Slope-Space Integrals for Specular Next Event Estimation

Monte Carlo light transport simulations often lack robustness in scenes containing specular or near-specular materials. Widely used uni- and bidirectional sampling strategies tend to find light paths involving such materials with insufficient probability, producing unusable images that are contaminated by significant variance. This article addresses the problem of sampling a light path connecting two given scene points via a single specular reflection or refraction, extending the range of scenes that can be robustly handled by unbiased path sampling techniques. Our technique enables efficient rendering of challenging transport phenomena caused by such paths, such as underwater caustics or caustics involving glossy metallic objects. We derive analytic expressions that predict the total radiance due to a single reflective or refractive triangle with a microfacet BSDF and we show that this reduces to the well known Lambert boundary integral for irradiance. We subsequently show how this can be leveraged to efficiently sample connections on meshes comprised of vast numbers of triangles. Our derivation builds on the theory of off-center microfacets and involves integrals in the space of surface slopes. Our approach straightforwardly applies to the related problem of rendering glints with high-resolution normal maps describing specular microstructure. Our formulation alleviates problems raised by singularities in filtering integrals and enables a generalization of previous work to perfectly specular materials. We also extend previous work to the case of GGX distributions and introduce new techniques to improve accuracy and performance.

Physics-informed machine learning for reduced-order modeling of nonlinear problems

Jan Sickmann Hesthaven, Qian Wang

A physics-informed machine learning framework is developed for the reduced-order modeling of parametrized steady-state partial differential equations (PDEs). During the offline stage, a reduced basis is extracted from a collection of high-fidelity solutions and a reduced-order model is then constructed by a Galerkin projection of the full-order model onto the reduced space. A feedforward neural network is used to approximate the mapping from the physical/geometrical parameters to the reduced coefficients. The network can be trained by minimizing the mean squared residual error of the reduced-order equation on a set of points in parameter space. Such a network is referred to as physics-informed neural network (PINN). As the number of residual points is unlimited, a large data set can be generated to train a PINN to approximate the reduced-order model. However, the accuracy of such a network is often limited. This is improved by using the high-fidelity solutions that are generated to extract the reduced basis. The network is then trained by minimizing the sum of the mean squared residual error of the reduced-order equation and the mean squared error between the network output and the projection coefficients of the high-fidelity solutions. For complex nonlinear problems, the projection of high-fidelity solution onto the reduced space is more accurate than the solution of the reduced-order equation. Therefore, higher accuracy than the PINN for this network - referred to as physics-reinforced neural network (PRNN) - can be expected for complex nonlinear problems. Numerical results demonstrate that the PRNN is more accurate than the PINN and both are more accurate than a purely data-driven neural network for complex problems. During the reduced basis refinement, before reaching its accuracy limit, the PRNN obtains higher accuracy than the direct reduced-order model based on a Galerkin projection.

2020

Physics-informed machine learning for reduced-order modeling of nonlinear problems

Jan Sickmann Hesthaven, Qian Wang

A reduced basis method based on a physics-informed machine learning framework is developed for efficient reduced-order modeling of parametrized partial differential equations (PDEs). A feedforward neural network is used to approximate the mapping from the time-parameter to the reduced coefficients. During the offline stage, the network is trained by minimizing the weighted sum of the residual loss of the reduced-order equations, and the data loss of the labeled reduced coefficients that are obtained via the projection of high-fidelity snapshots onto the reduced space. Such a network is referred to as physics-reinforced neural network (PRNN). As the number of residual points in time-parameter space can be very large, an accurate network – referred to as physics-informed neural network (PINN) – can be trained by minimizing only the residual loss. However, for complex nonlinear problems, the solution of the reduced-order equation is less accurate than the projection of high-fidelity solution onto the reduced space. Therefore, the PRNN trained with the snapshot data is expected to have higher accuracy than the PINN. Numerical results demonstrate that the PRNN is more accurate than the PINN and a purely data-driven neural network for complex problems. During the reduced basis refinement, the PRNN may obtain higher accuracy than the direct reduced-order model based on a Galerkin projection. The online evaluation of PINN/PRNN is orders of magnitude faster than that of the Galerkin reduced-order model.

2021

Light Path Gradients for Forward and Inverse Rendering

Tizian Lucien Zeltner

Physically based rendering is a process for photorealistic digital image synthesis and one of the core problems in computer graphics. It involves simulating the light transport, i.e. the emission, propagation, and scattering of light through a virtual scene that is defined by a detailed description of object geometry and appearance. Research over the last decades has led to sophisticated rendering techniques and recently, the inversion of this process, i.e. recovering scene parameters from image observations, has also received significant attention. In this thesis, we investigate methods in both physically based forward and inverse rendering that exploit light path gradients.The first part is concerned with scattering from specular surfaces, which produces complex optical effects that are frequently encountered in realistic scenes: intricate caustics due to focused reflection, multiple refractions, and high-frequency glints from specular microstructure. Yet, despite their importance and considerable research to this end, sampling of light paths that cause these effects remains a formidable challenge of forward rendering.We propose a surprisingly simple and general path sampling strategy that targets the examples above. Valid light path configurations need to fulfill the physical laws of reflection and refraction, and we find these using a numerical root-finding process that is driven by geometric light path gradients. In contrast to prior work, our method supports high-frequency normal- or displacement-mapped geometry, samples specular-diffuse-specular (SDS) paths, and is compatible with standard Monte Carlo methods including unidirectional path tracing. We demonstrate our method on a range of challenging scenes and evaluate it against state-of-the-art methods for rendering caustics and glints.In the second part, we consider differentiable rendering algorithms. These propagate derivatives through the full light transport simulation to solve inverse rendering problems via gradient-based optimization. Recent progress has led to methods that can simultaneously compute derivatives with respect to millions of scene parameters. At the same time, elementary properties of these methods remain poorly understood.Current algorithms for differentiable rendering are constructed by mechanically differentiating a given primal algorithm. As differentiation fundamentally changes the underlying problem, this is often suboptimal and instead, primal and differential algorithms should be decoupled so that the latter can suitably adapt. This is surprisingly complex. Even the most basic Monte Carlo path tracer already involves several design choices concerning the techniques for sampling materials and emitters, and their combination, e.g. via multiple importance sampling (MIS). Differentiation causes a veritable explosion of this decision tree: should we differentiate only the estimator, or also the sampling technique? Should MIS be applied before or after differentiation? Are specialized derivative sampling strategies of any use? How should visibility-related discontinuities be handled when millions of parameters are differentiated simultaneously? We provide a taxonomy and analysis of different estimators for differential light transport to provide intuition about these and related questions.

EPFL2021

Light Path Gradients for Forward and Inverse Rendering

Tizian Lucien Zeltner

EPFL2021

Physics-informed machine learning for reduced-order modeling of nonlinear problems

Jan Sickmann Hesthaven, Qian Wang

2020