How to use PSDR-CUDA for forward rendering has been shown in the previous section. In what follows, we focus on performing differentiable rendering.
Similar to the forward-rendering case, after being loaded and configured, a scene can be rendered in a differentiable fashion by calling an integrator’s renderD()
method. The return value of this method is an Enoki CUDA array of the type enoki.cuda_autodiff.Vector3f
.
The following example generates a derivative image image_deriv
with respect to the rotation angle P
of the first mesh about the Z-axis.
import enoki as ek
from enoki.cuda_autodiff import Float32 as FloatD, Vector3f as Vector3fD, Matrix4f as Matrix4fD
import psdr_cuda
# Load the scene without configuring it
scene = psdr_cuda.Scene()
scene.load_file('scene.xml', auto_configure=False)
# Initialize a parameter P for differentiation
P = FloatD(0.)
ek.set_requires_gradient(P)
# Construct a rotation matrix
mat = Matrix4fD.rotate(Vector3fD(0., 0., 1.), P)
# Left-multiply this matrix to the to-world transformation of the first mesh of the scene
scene.param_map["Mesh[0]"].append_transform(mat)
# Configure the scene
scene.configure()
# Start rendering!
image = psdr_cuda.DirectIntegrator().renderD(scene, sensor_id=0)
# Compute derivative image with respect to P using forward-mode autodiff
ek.forward(P)
image_deriv = ek.gradient(image)
What follows is another example that computes the gradient of the image RMSE loss
, given a target image target_image
, with respect to all vertex positions of Mesh[0]
. The resulting grad
can then be used in gradient-based optimization pipelines to minimize loss
.
import enoki as ek
from enoki.cuda_autodiff import Float32 as FloatD, Vector3f as Vector3fD, Matrix4f as Matrix4fD
import psdr_cuda
# Load the scene without configuring it
scene = psdr_cuda.Scene()
scene.load_file('scene.xml', auto_configure=False)
# Compute gradient with respect to mesh vertex positions
ek.set_requires_gradient(scene.param_map["Mesh[0]"].vertex_positions)
# Configure the scene
scene.configure()
# Start rendering!
image = psdr_cuda.DirectIntegrator().renderD(scene, sensor_id=0)
# Compute the RMSE image loss
loss = ek.sqrt(ek.hmean(ek.squared_norm(target_image - image)))
# Reverse-mode autodiff
ek.backward(loss)
# Obtain the gradient of the loss
grad = ek.gradient(scene.param_map["Mesh[0]"].vertex_positions)