| EQUIPE ASSOCIEE |
CIPRus |
sélection |
2009 |
| Equipe-Projet INRIA : ARTIS | Organisme étranger partenaire : Cornell University |
| Centre de recherche INRIA : Grenoble Rhône-Alpes Thème INRIA : CogD |
Pays : United States of America |
Coordinateur
français |
Coordinateur
étranger |
|
| Nom, prénom | Holzschuch, Nicolas | Bala, Kavita |
| Grade/statut | CR1 INRIA | Assistant Professor |
| Organisme d'appartenance (précisez le département et/ou le laboratoire) |
INRIA | Cornell University, department of Computer Science |
| Adresse postale | 655 Avenue de l'Europe Inovallée 38334 St-Ismier CEDEX France |
4130 Upson Hall Cornell University Ithaca, NY 14853-7501 USA |
| URL | http://artis.inrialpes.fr/ | http://www.cs.cornell.edu/~kb |
| Téléphone | +33 476-615-506 | +1 607-255-7316 |
| Télécopie | +33 476-615-440 | +1 607-255-4428 |
| Courriel | Nicolas.Holzschuch@inria.fr | kb@cs.cornell.edu |
La proposition en bref
|
Titre de la thématique de collaboration (en
français et en anglais) :
|
| Descriptif (environ 10 lignes) :
Photorealistic rendering deals with the production of pictures of virtual worlds that are as close as possible to what a real photograph of this virtual world would look like. Considerable progress has been made in recent years, and photorealistic pictures are being used in severa sectors of the industry: virtual prototyping, special effects for motion picture, video games... However, truly photorealistic pictures of a virtual world are still difficult to get. There are multiple difficulties to overcome: model acquisition, model representation, scalability, sampling and perceptual issues. Our goal in this project is to address all these issues simultaneously, targetting the production of high-quality photographic like pictures that are capable of passing a "Turing-test": they are impossible to separate from photographs of the real world, with all its complexity. Our goal in this project is to address the many hard challenges remaining in Photorealistic Rendering, especially dealing with the inclusion of real-world objects in virtual scenes and modelling complex materials, such as low-order scattering or high-reflectance materials. The challenges we selected have two points in common: they're regarded as difficult research challenges, and they would greatly enhance the realism of the pictures generated. Both teams stand to gain from a joint work in this area. This joint work should result in several scientific breakthroughs, with the production of photorealistic pictures of highly complex virtual worlds. |
Présentation
détaillée de l'Équipe Associée
1. Objectifs scientifiques de la proposition (1
à 2 pages)
Challenges in Photorealistic Rendering
Photorealistic rendering deals with the production of pictures of virtual worlds that are as close as possible to what a real photograph of this virtual world would look like. Considerable progress has been made in recent years, and photorealistic pictures are being used in severa sectors of the industry: virtual prototyping, special effects for motion picture, video games...
However, truly photorealistic pictures of a virtual world are still difficult to get. There are multiple difficulties to overcome: model acquisition, model representation, scalability, sampling and perceptual issues. Our goal in this project is to address all these issues simultaneously, targetting the production of high-quality photographic like pictures that are capable of passing a "Turing-test": they are impossible to separate from photographs of the real world, with all its complexity.
Our goal in this project is to address the many hard challenges remaining in Photorealistic Rendering, especially dealing with the inclusion of real-world objects in virtual scenes and modelling complex materials, such as low-order scattering or high-reflectance materials. The challenges we selected have two points in common: they're regarded as difficult research challenges, and they would greatly enhance the realism of the pictures generated. Both teams stand to gain from a joint work in this area. This joint work should result in several scientific breakthroughs, with the production of photorealistic pictures of highly complex virtual worlds.
There are several areas of photorealistic rendering that are currently regarded as significant research challenges by the graphics community:
- the inclusion of real world objects in virtual scenes, with lighting exchanges going in both directions,
- modelling complex materials, such as inhomogeneous materials with scattering properties or special reflectance functions,
- building hierarchical representations for complex geometrical models, where the higher-level representations account for the micro-scale variations at the most accurate representations,
- scalability in itself, that is the ability to run global illumination simulations on really large scenes, with million of polygons and tens of thousands of light sources,
- perceptual issues: our photorealistic pictures will, eventually, be judged by a human viewer, with his own human perception; some details of the model will be perceived with more accuracy than others. We need to focus on these details.
- highly reflective environments: for some objects, the ratio of the incoming energy with the outgoing energy is close to one: almost all the energy is being reflected. For example, at the surface of some white clouds, there is almost total reflection, resulting in this bright white aspect. And inside diamond gems, there is almost total inside reflection. These environments are a challenge for existing global illumination simulation methods, with a very slow convergence.
All these issues are deeply inter-connected; for example, building a hierarchical representation for a complex object can be done by replacing micro-scale surface details by macro-scale and a complex material model, in a manner similar to the definition of the Torrance BRDF model. Such a hierarchical representation can be validated by perceptual criteria, and replacing a complex object with a higher level definition is a way to deal with scalability issues. Similarly, the inclusion of real world objects in virtual scenes is currently a difficult problem because the acquisition process gives highly detailed objects, with sub-mm details. Building a hierarchical representation for these objects is a prerequisite for including them in global illumination simulations.
These issues are being regarded by the research community as interesting, but too complex to be solved with existing algorithms and technologies. Our common ambition is to target all these issues, with the long term goal of producing photorealistic pictures of really complex worlds.
Work-program for the duration of the EA
Complex materials and Hierarchical representations: First, we will focus on the twin problems of hierarchical representations for complex objets and dealing with complex material properties. Taking as input a complex object, e.g. obtained through 3D scanners, highly detailed at the micro level, our goal is to build a lower level representation, with geometric details being averaged. However, we wish to keep the light-reflecting properties of the object, both for its visual aspect and for its interaction with the virtual scene. This requires building a reflectance model that behaves, at a larger scale, like the average of the micro-scale details in the most accurate geometry. Simultaneously, we will be working on complex material models, including sub-surface scattering and inhomogeneous materials. These two research themes are deeply inter-connected: the algorithms developed through the latter will help in illumination simulations for the reflectance models created with the former.
Scalability and Perceptual issues: Next, we will target the connected issues of scalability and perceptual rendering. Our goal is to design illumination simulation algorithms that are capable to handle really large scenes, with millions of polygons and thousands of light sources. Our work will build on existing work on Frequency Analysis of Light Transport (by the ARTIS team) and multi-dimensional light cuts (by Cornell).
High-Reflectance Materials Finally, we will focus on the specific questions of convergence of existing methods in the presence of highly reflective environments, or surfaces in close proximity to one another. In these environments, existing simulation methods converge slowly, with a large variange, or even diverge. Hence the need to work out new simulation methods that are able to handle these complex cases.
2. Présentation
des partenaires (1 page environ par partenaire)
Présentez
les différentes équipes
participantes ;
Donnez, pour
chaque partenaire, la liste des chercheurs impliqués dans la proposition
ainsi qu'un bref CV du responsable ;
Remarque : pour chaque partenaire, la proposition peut être coordonnée
par un jeune chercheur à condition de démontrer l'investissement
de l'équipe
à laquelle il appartient ;
Indiquez,
pour chaque partenaire, les étudiants impliqués dans la proposition.
Donnez une estimation de leur nombre et précisez si des thèses
en cotutelle sont prévues ;
Présentez l'historique
de la collaboration entre les équipes ;
Insérez
des liens vers les pages des personnes, laboratoires,
organismes....
"ARTIS" (Acquisition, Representation and Transformations for Image Synthesis):
ARTIS is both a project-team of INRIA Rhône-Alpes and a team of the LJK laboratory, a joint research unit of CNRS, INPG and UJF.
The project was created in January, 2003, based on the observation that classical image synthesis methods appear limited with respect to the variety of current applications. In particular, the typical approach consisting in separately modeling a 3D geometry and a set of photometric properties (reflectance, lighting conditions), then in computing their interaction to produce a picture, is too restrictive. First, this approach severely limits the adaptability to particular constraints or freedom allowed in each application (such as precision, real-time, interactivity, uncertainty about input data...); second, it restricts the possible image classes and does not easily lend itself to new uses in a more "expressive" way, such as for illustration, where a form of hierarchy on image constituents must be constructed.
One of the goals of the project is the definition of a more generic framework for the creation of synthetic images, integrating elements of 3D geometry, of 2D geometry (built from these), of appearance (photometry, textures...), of rendering style, and of importance or relevance for a given application. The ARTIS project therefore deals with multiple aspects of image synthesis: model creation from various sources of data, transformations between these models, rendering and imaging algorithms, and the adaptation of the models and algorithms to various constraints or application contexts.
The manager for the EA will be Nicolas Holzschuch. He is a CR1 researcher at INRIA. He has done his Ph. D. in the iMAGIS/GRAVIR research team in 1996. He has recently completed a sabbatical stay of 12 month at Cornell University. His work concern various aspects of global illumination simulation, using hierarchical finite element methods, such as wavelets, as well as exploiting possibilities of GPU for real-time realism. He has several contributions on using Frequency Analysis for more efficient computations of global illumination. He organised the 2007 Eurographics Symposium on Rendering in Grenoble, and was a member of the Program Committee of Eurographics Symposium on Rendering 2006 to 2008 and Eurographics 2004 and 2009.
Inside ARTIS, Cyril Soler (CR1 INRIA), Fabrice Neyret (CR1 CNRS) and Kartic Subr (post-doc INRIA) will also play a major role in the collaboration. The following 5 doctoral students are also involved: Charles de Rousiers, Nassim Jibaï, Cyril Crassin, Hedlena Bezerra and Adrien Bousseau.
Cornell University:
The Program of Computer Graphics, established in 1973, has been one of the pioneering leaders in computer graphics research in the world. Since its inception, the Program has been well-known for pioneering research in computer graphics using a rigorous scientific basis combined with experimental verification. The program combines a very strong interdisciplinary team of faculty, from the fields of physics, computer science, mechanical and electrical engineering, applied mathematics, and perception psychology, all who bring a wide range of experience and innovative perspectives to address complex research computer graphics problems.
In the recent past, one of their major goals has been to create realistic simulations which are physically accurate and perceptually indistinguishable from real-world scenes. Their research spans a broad range of topics to achieve this goal. They work on all three areas of physically-based rendering: the physics of light-reflection models, the computation and thermodynamics of energy propagation, and the creation of models of the human visual perceptual systems.
Their collective publishing record is among the strongest in the field, and they have led many workshops at ACM SIGGRAPH and at Cornell in the areas of measurement, rendering, and perception. Many of our alums have continued in academia and have held professorial appointments at CalTech, Utah, Georgia Tech, MIT, Stanford, Carnegie Mellon, University of Oregon, and UC Berkeley, to name a few. Others have gone on to become leaders in the fields of computer graphics, computer animation, and computer-aided-design in Architecture. Former students from the Program of Computer Graphics have now won five of Hollywood's Technical "Oscars", one actual "Oscar", and five have won the prestigious SIGGRAPH Achievement Award.
Kavita Bala is an Assistant Professor in the Computer Science Department and Program of Computer Graphics at Cornell University. She received her S.M. and Ph.D. from the Massachusetts Institute of Technology (MIT), and her B.Tech. from the Indian Institute of Technology (IIT, Bombay). Bala leads research projects in scalable rendering, perceptually-based rendering, interactive global illumination, parallelization, and texture synthesis. She has co-authored the graduate-level textbook "Advanced Global Illumination", now in second edition. In 2005 she co-chaired the Eurographics Symposium on Rendering (EGSR). She has served on more than 25 program committees including SIGGRAPH, SIGGRAPH Asia, Eurographics, Eurographics Symposium on Rendering, Interactive Ray Tracing, Interactive 3D Graphics, Graphics Interface. She is also the associate editor of IEEE TVCG Transaction on Visualization and Graphics. Bala has received the NSF CAREER award, and Cornell's College of Engineering James and Mary Tien Excellence in Teaching Award.
Bruce Walter is a research associate at Cornell University. He spent 12 months as a post-doctoral researcher with the iMAGIS team, a predecessor of ARTIS, in Grenoble, France in 1998-99. His research areas include global illumination, material modeling, and perceptual rendering, and has served on the program committes for SIGGRAPH (2008), Eurographics Rendering Symposium (2003/5/6/7/9) and IEEE Symposium on Interactive Ray Tracing (2006/7/8).
Historique de la collaboration entre les équipes:
- François Sillion, the former manager of the ARTIS team, spent 24 months as a post-doc student at the Program of Computer Graphics, Cornell University, in 1990-1991.
- Bruce Walter (Cornell University) spent 12 months in the iMAGIS team in Grenoble in 1998-1999 (the iMAGIS team was split in 2001 into two teams: ARTIS and EVASION).
- Nicolas Holzschuch (ARTIS) spent 12 months in 2007-2008 at Cornell University, in a Sabbatical stay funded by INRIA. This EA proposal is built, in part, on the research themes that were started in common during this Sabbatical.
3. Impact
(1 page maximum)
Indiquez l'impact de cette collaboration
sur :
- les objectifs scientifiques des équipes participantes
;
- les relations entre les partenaires et entre les instituts (par
exemple discutez de la complémentarité, de la similarité pour
un effet de masse
critique, de la répartition des tâches pour un gros développement,
etc.)
ARTIS and Cornell University are both working at the foreground of research on photorealistic rendering, and have a long experience on the subject. Their skills are complementary. This Associate Research Team targets hard and difficult research areas that are probably out of reach for each team separately. Joining forces will allow them to effectively target these areas, and attain significant results faster. We are expecting that this collaboration will result in regular publications, every year. Some of these cooperations are already under way, as a consequence of Nicolas Holzschuch's sabbatical stay at Cornell, but the potentials for synergy are much larger.
4. Divers : toute autre information que vous jugerez utile d'ajouter.