A Brief Overview of Lighting in Unity 5

Entire books could be written about lighting in Unity 5, but here is a brief mishmash of notes from the past few months of work in Unity 5. I’ve split it into 6 sections:

Most of this is applicable to any content made with Unity, not just virtual reality applications. This is a fairly high-level overview but it does assume some prior knowledge of basic graphics & lighting concepts. Let’s begin!

 Forward vs. Deferred Rendering Paths

When in doubt, go with a forward rendering path. It’s ideal for VR applications for several reasons. Forward rendering has a low upfront performance cost, it’s easier on the CPU with fewer draw calls, it runs well on mobile VR devices, and it supports anti-aliasing as well as translucent materials. Here’s a quick overview to help you decide which pipeline is right for your application:

 Deferred Rendering

Deferred rendering has a very high upfront performance cost but it’s a fixed performance cost, allowing you to add a ton of geometry and a ton of lights without much extra cost. Do you have hundreds of small realtime lights? Go with a deferred path. The cost of a deferred render is proportional to the number of pixels that a light hits (the light volume). Scene complexity (number of objects that a light hits) does not affect performance. Sidenote: UE4 only has a deferred rendering pipeline, which is one reason why many VR demos made with UE4 run slowly on old PCs.

 Benefits

 Tradeoffs

 Forward Rendering

Forward rendering has a low upfront fixed cost, and thus is best for low complexity scenes, but its performance cost scales up with complexity. Does your scene only have a couple realtime lights? Go with a forward path. The cost of a forward renderer is proportional to the number of lights in the scene. Few pixel lights = fast performance.

 Tradeoffs

 Benefits

When deciding between a forward or deferred renderer, keep in mind that we are only discussing realtime lights. You can have as many baked lights as you wish in your scene. Bake your lightmap, and disable them before runtime! At runtime, the renderer only considers realtime lights in your scene.

 Realtime vs. Baked lighting

If you care about realistic shadows, bake the light. Hard/soft is sufficient for some, but looks horrible in many cases. (example: spotlight over a table) With a realtime light, you see a blocky shadow underneath. With baked lights, you see a soft, diffused shadow under the table). Real-time lighting is only needed if change something about the light (color, position, etc.). Baked lighting produces higher quality results & reduces performance requirements. Especially with the constraints of VR, baked light should be used much as possible. Baked lighting & shadows don’t change to realtime dynamic object movement (ex. of player movement) but tricks can compensate (see “light probes” below).

 Improving Baked Lightmap Quality

When baking, you can improve lightmap quality by increasing the baked resolution slider (40-100 texel resolution is reasonable), cranking up the ambient occlusion and enabling final gather. Before baking, check your project’s Quality & Player settings. In the Quality tab, use hard and soft shadows with very high resolution. In the Player tab, select the deferred renderer. You should disable precompute GI before you bake. Even if you use lower quality shadows or a forward renderer at runtime, use these settings during lightmap baking for best results. You can always switch back to forward rendering or lower graphics settings after you’re done baking.

 Engine Lighting Tools

 Area light

 Point light

 Spot light

 Directional light

 Ambient light

 Global illumination (GI)

Global illumination calculates indirect bounced lighting & shadows from (direct) light sources for more realistic scenes. Realtime GI allows for a changing light scene. i.e. changing color of a light, moving the directional “sun” light, changing from day to night, etc. To increase the quality of realtime GI, increase the precompute resolution.

 Light Probes

If you’re using baked lighting, dynamic objects won’t have accurate shadows or lighting information. They’ll appear dull. Use light probes to add realistic lighting, shadowing & coloring for dynamic objects. It’s a precomputed volume of light information that can be applied to objects when the move in and out of areas, simulating realtime lighting cheaply. Here’s a quick tutorial on how to place a good volume of light probes in your scene.

Light probes store light information with a Spherical Harmonics (SH) model. The forward renderer’s base pass renders a pixel light as well as all SH lights (for free). Thus, each light probe has very little cost on the CPU and incurs no cost at all on the GPU. Read more about this here.

 Reflection Probes

For a reflective surface like a mirror or glossy screen, you will need to see surrounding objects reflected on the surface. This can be cheaply achieved with a reflection probe. Unfortunately they can only be cube shaped at the moment, and work best for cube shaped rooms. For scenes with dynamic objects, you can use realtime reflection probes. For static scenes (or to save performance), you can use baked reflection probes.

bpcem1.jpg
Reflection probes use a box projection to accurately model reflections such as a window over a marble floor. A cubemap alone won’t have accurate reflects, a box mapped reflection will fix this. Custom cubemaps can be made for more realistic “wavy” reflections (ex. on marble flooring).

 Emissive lighting

In the case of a lamp & a lampshade, an emissive texture on the lampshade is needed. It will make the lampshade appear bright, and shine soft light on the surroundings. A point light on the inside is also needed to shine harsher, brighter light through the top and bottom of the lamp. It should get occluded by the lampshade, and thus light will exit the lamp in an double cone shape. An area light will have better shadows, but an emissive map can be shaped light (and thus shaped reflection)

 Lighting Scenarios

 Ceiling lights (tube/box lights)

Strip and box lights with accurate, soft light can be achieved with the emissive property on the Standard Shader. You can use an area light or an emissive material, or a mix of both. A tube light on the ceiling should probably glow brightly when you look at it: use an emissive material on the tube light’s individual tubes. The actual light produced by this tube light can be done with an area light, providing more shadows with nicer penumbra.

 Windows

When far from a window, exposure control on the scene and limited eye dynamic range means the interior should be exposed correctly but the window should be blown out, showing a bright light with soft shadows. This can be achieved with tone mapping HDR to LDR. Watch this tutorial. As the player walks from a dark indoor scene to the bright outdoors, the scene exposure should change as well.

Windows with colored textures (ex. church windows) can use cookies to project the patterns on a floor, but cookies are greyscale on the alpha channel with no color data. This is tricky and some hacks are required (out of the scope of this blog post)

 Cloudy day

In a cloudy outdoors scene, the directional “sun” light should be soft & diffused after scattering through a cloud layer. Directional lights are probably inappropriate as they create harsh shadows.Consider using gray-ish ambient light, a cloudy skybox cubemap and point lights for moody lights and shadows.

 Crepuscular Rays

Also known as sun rays or light shafts, these look great in VR, especially with floating dust particles. They are computationally expensive but Robert Cupisz is working on a performant light shaft implementation. There are many more solutions on the Unity Asset store but I haven’t tried them so I have no recommendations at this time.

 Improving Performance

Here are a few performance tricks:

 Sidenote: Flickering lights in VR?

If one of your lights flickers in one eye (left/right VR camera), Unity is culling out the light. I think there’s a bug here? Solution: Mark the light as Important and the engine won’t disable it. Remember you can also programmatically enable/disable lights as needed in a scene, giving you full control.

 Post Processing & Level Design

When designing realistic lighting, have scenes with high contrast: areas that are bright & well lit, correct shadows, and also areas that are dark. Don’t evenly light a whole scene (ex. just ambient light), or it will appear flat, game-like and fake.

 Lookup Textures (LUTs)

asdf.png
Left: neutral Right: standard “ConstrastEnhanced” texture

 Further Reading

Next time, I’d like to cover a few more topics in depth: simulating IES with cookies, anti-aliasing (MSAA, TAA) and more. I wrote this several months ago when I was first getting started with Unity. If you come across any inaccuracies, please let me know.

If you enjoyed this blog post, you should follow me on Twitter: @dshankar

 
319
Kudos
 
319
Kudos

Now read this

12 performance tricks for optimizing VR apps in Unity 5

VR applications are far more computationally intensive than non-VR counterparts, making performance optimization a crucial task. If you’re targeting the mobile devices like the GearVR, this becomes even more paramount. These are... Continue →