Lumen: Global Illumination in Unreal Engine

"Real-time global illumination has always been a holy grail of computer graphics."

Lumen is a dynamic global illumination and reflections system implemented in Unreal Engine (UE) 5. Lumen Global Illumination solves diffuse indirect lighting. For example, light bouncing diffusely off a surface picks up the color of that surface and reflects the colored light onto other nearby surfaces, creating an effect called color bleed. Meshes in scenes also block indirect lighting, which also produces indirect shadowing. The magic of Lumen is its ability to simulate infinite diffuse bounces in real time. In this project, we aim to explore how Lumen realizes real-time global illumination and improve the rendering efficiency without compromising too many visual effects.

Introduction and Problem Description

Lumen is a dynamic global illumination and reflections system implemented in Unreal Engine (UE) 5. Lumen Global Illumination solves diffuse indirect lighting. In this final project, we aim to explore and implement the core and essential idea behind Lumen. Ignoring the complex engineering and framework limitations, we want to build a global illumination renderer that traces the ray to collect direct light, but uses a Radiosity-based method for accumulating indirect light as Lumen does. Our goal is to achieve an amazing global illumination quality faster in a real-time rendering fashion.

Solving the Discretized Rendering Equation

In general, the difficulty of real-time global illumination lies in solving the rendering equation:

By discretizing the equation and breaking the continuous surface into surface elements, we are able to reach the following linear equations:

where \( B_i(u) \) represents the radiance of the \( i \)-th surface element at direction \( v \), \( B_{e,i} \) is the emission from the \( i \)-th surface element.

Now the question is how to efficiently solve this linear equation. One way is to use Gaussian elimination, but the time complexity goes up in cubic order with respect to the number of surface elements.

Lumen's Approach: Reuse

Lumen incorporates two new ideas: surface and radiance cache. They sample and store some radiance \( B_i(v) \) of the \( i \)-th surface element and the corresponding material information at the \( t \)-th time step, and use them to update the \( B \)s at the \( t+1 \)-th time step.

This method relies on one condition to converge quickly and accurately to the true global illumination solution: the radiance of the \( i+1 \)-th surface element at the \( t \)-th time step does not differ significantly from itself at the \( t+1 \)-th time step. And this condition is generally true in scenarios where real-time rendering is needed.

Overview of the Methods and Progress

Ray Marching and Mesh Distance Fields

While the previously introduced equation provides the theoretical foundation for real-time global illumination, our implementation experience reveals that the core strength of Unreal Engine 5's Lumen system lies primarily in its engineering design. Several key challenges must be addressed to enable practical performance, among which the first is:

• Assuming unlimited parallelism, Equation \( B_i^{t+1}(u) = B_{e,i}(u) + \rho_i \sum_{j=1}^{N} \sum_v F_{ij}(u,v) \cdot B_j^t(v) \) reduces the computational complexity of solving the discretized rendering equation \( B_i(u) = B_{e,i}(u) + \rho_i \sum_{j=1}^n \sum_v F_{ij}(u,v) B_j(v) \) to \( O(N) \), where each thread independently updates a single \( B_i^{t+1} \). However, for scenes containing a large number of triangles, this complexity remains insufficient for real-time performance.

Lumen addresses this challenge by casting a set of rays from each surface element. For the \( i \)-th surface element, rays are cast to identify contributing \( B_j^t \) values, compute the associated form factors \( F_{ij} \), and update \( B_i^{t+1} \). This strategy effectively approximates the full summation by randomly sampling a subset of contributing elements rather than exhaustively iterating over all possible surface pairs. Given infinite parallelism, this approach reduces the complexity to \( O(\text{rays} \times O(\text{ray-surface intersection})) \).

The second engineering challenge is as follows:

• Conventional ray-triangle intersection acceleration structures, such as bounding volume hierarchies (BVHs), are computationally expensive. According to Wright et al., real-time performance using such structures is limited to approximately half a ray per pixel. Additionally, tree-based data structures like BVHs exhibit poor performance on GPUs due to their branch-heavy execution patterns.

To overcome this, Lumen avoids direct ray-triangle intersection and instead employs ray marching accelerated by signed distance fields (SDFs), which are significantly more GPU-friendly.

Implementation Progress 1: World-Space SDF

We began by implementing a world-space SDF using a 3D texture that encapsulates the entire scene’s bounding box, with a resolution of \( 256 \times 256 \times 256 \). The SDF is computed and updated in real time using the Jump Flooding Algorithm (JFA). The result of this initial implementation is illustrated in Figure 1.

However, this implementation exhibits several limitations. Achieving adequate geometric detail necessitates high-resolution SDFs, yet JFA only provides an approximation, which introduces imprecision at finer resolutions. As shown in Figure 1, some voxels remain incorrectly updated, resulting in noticeable artifacts (e.g., white pixels indicating rays that fail to intersect geometry). Furthermore, the performance is suboptimal; rendering the hit point lengths at this resolution yields only ~25 FPS.

Figure 1a: Our world-space SDF at resolution \(256 \times 256 \times 256\).

Figure 1b: UE5 Mesh SDF with resolution \(56 \times 49 \times 56\).

Upon reviewing documentation and visuals from Unreal Engine 5, we identified Figure 1b, which shows a mesh SDF implementation with a much lower resolution of \(56 \times 49 \times 56\). This observation led us to hypothesize that Lumen maintains a separate SDF for each mesh, computed in the object space and covering only the mesh’s bounding volume. During ray marching, the bounding box of the mesh is first intersected, and marching proceeds in object space to find the hit point on the mesh surface. This insight is consistent with Lumen’s use of the term Mesh Distance Field.

Implementation Progress 2: Mesh-Space SDF

Based on this understanding, we implemented mesh-space SDFs. Each mesh maintains its own distance field within object space. This revision significantly improved both accuracy and performance, achieving approximately 160 FPS. The result is shown in Figure 2. Minor visual artifacts, such as aliasing, are attributable to the lack of trilinear filtering in our SDF sampling.

Figure 2: Our mesh-space SDF implementation (without trilinear filtering).

Surface and Radiance Caching Using Cards

Having established a method to compute surface-ray intersections via signed distance field (SDF) ray marching, we encounter the third key challenge:

• Ray marching using SDFs yields only the spatial coordinates of the surface intersection, but it does not provide information about the corresponding surface element or the material properties at the hit point. In other words, we cannot directly obtain \( F_{ij} \) and \( B_j^t \) in Equation \( B_i^{t+1}(u) = B_{e,i}(u) + \rho_i \sum_{j=1}^{N} \sum_v F_{ij}(u,v) \cdot B_j^t(v) \).

An initial idea was to allocate a texture with the same resolution as the mesh’s SDF to store surface properties. However, this approach quickly proved impractical. The primary reason is that the discretized rendering equation assumes a purely diffuse scene, wherein only radiosity must be stored. For more complex materials—such as mirrors—radiance samples over the hemisphere must be retained. For instance, if each surface element casts 64 rays to sample hemispherical radiance, and each sample stores a direction (3 floats), probability (1 float), and value (3 floats), this results in 1792 bytes per surface element. Given Lumen's maximum supported SDF resolution of \(128 \times 128 \times 128\), the total memory requirement would exceed 3.5 GB per object, which is prohibitively large.

Figure 3: UE5 surface cache structure using "Cards".

Figure 3, sourced from the official Unreal Engine documentation, reveals Lumen’s approach to mitigating this storage overhead. The key observation is that the axis-aligned bounding box of a mesh typically contains substantial regions of empty space that do not correspond to the actual surface and thus need not be cached.

To avoid caching this non-surface volume, Lumen introduces the concept of Cards. A card is a spatial cuboid that is discretized to store surface data. A given mesh is decomposed into multiple cards, each capturing a portion of its surface. This subdivision reduces the amount of unused space represented in the cache. However, it introduces an additional lookup cost—during ray intersection, one must identify the specific card containing the hit point to access the corresponding surface data. In the limiting case where a mesh uses a single card, the scheme degenerates into a full-object-space cache with resolution equal to that of the SDF. Thus, the number of cards offers a tunable tradeoff between memory efficiency and access performance.

Implementation Progress 3: Card Generation

To enable card-based caching, we implemented a mesh decomposition strategy using the K-Means clustering algorithm to partition the surface point cloud into a specified number of clusters (equal to the number of desired cards). For each cluster, we computed an oriented bounding box (OBB) using principal component analysis (PCA). The resulting card structure is visualized in Figure 4a. In parallel, we also implemented trilinear interpolation for SDF sampling to improve ray-marching precision.

Implementation Progress 4: Surface Normal Cache

Following card generation, we implemented a surface normal cache. This cache stores the normal vectors corresponding to the discretized surface points within each card. A visualization of the resulting normals is provided in Figure 4b. The visualization frame rate is now about 80 fps.

Figure 4a: Card generation for the CBbunny scene with a maximum of 12 cards.

Figure 4b: Visualization of cached surface normals.

Future Work: Surface and Radiance Cache Implementation

Our subsequent development efforts will focus on the following objectives:

1. Integration of Surface Texture Data: We aim to incorporate material-specific surface information into the cache, with initial support for two fundamental material types: diffuse and specular (mirror-like) surfaces.
2. Radiance Cache Implementation and Update Strategy: We plan to design and implement a radiance cache system, incorporating Lumen's temporal reuse strategy to efficiently update cached radiance values across frames.
3. Importance-Based Radiance Sampling: We will implement importance sampling techniques for radiance acquisition, enabling more accurate and efficient integration by prioritizing samples that contribute significantly to the lighting solution.

Resources