Programming Blog 11 – Occlusion Culling: The Great Battle

Hi guys!

Today I’d like to talk about the occlusion culling process used in Monstrum. As good as Unity is, we couldn’t use its culling support since it doesn’t support dynamic occlusion. Additionally, our light culling system was overhauled as well using a completely different system to the one we had before.


Geometry Occlusion

We tried everything. Well, maybe not everything, but we did go through quite a few techniques before we got to this one. We tried cheaply rendering the scene and processing the image to look for renderers. An improvement, but it had an unavoidable expensive function call to get the texture off the GPU. We tried culling with raycasts from the camera every frame to get the renderers. Too many raycasts were necessary to see everything. We tried culling our culling raycasts. It worked in smaller rooms, but as soon as we entered a larger room it somehow ended up doing even more raycasts than before. Fantastic.

Well, we now have a solution. Or at least a better solution. It isn’t perfect, but it breaks in places we should be able to avoid. I don’t know if this method already has a name, or if there is something that was overlooked that can make it more generic or something. But here goes.

The culling starts by setting up a grid encapsulating the ship. Each node on the grid represents a cuboid the size of a corridor piece, and stores information about what nodes it can see, and what renderers are inside it. This is used for storing the occlusion data. Each time the camera changes what node it is in, the node at the camera tells all the nodes it can see to enable the renderers associated with them. Simple. The tricky part is working out what nodes can see each other, and this is the bit that is new.

Each node raycasts up, down, left, right, forward and backwards to figure out whether it can see the node immediately around it. This creates a graph of the level that we can traverse. You might notice that raycasts won’t necessarily generate the most accurate graph. You would be right! It does have surprisingly few errors actually, but we will likely generate a much more accurate graph later to iron out the kinks.

Next, the core algorithm kicks in. To find out what nodes a given node can see, the graph is traversed using a flood-fill method, with a few modifications. Any node we expand in to is added to the list of nodes the camera node can see. The modifications are used to stop expansion in to nodes we won’t be able to see. We are currently experimenting with both real time and baked versions of this process.

  • If the angle between the expand direction and the camera’s forward is > 135° , disallow expansion. This forces the flood-fill to spread out vaguely in the direction of the camera’s forward – the bit we really want to see. (Real time only)

  • A node is prevented from expanding when its centre is outside the camera’s frustum by a small margin, since only stuff in the field of view will be rendered. (Real time only)

  • Nodes that are reached by going around a corner ‘too much’ are prevented from expanding. This check is not done in open areas, since corners won’t necessarily occlude.

  • Once the path goes right, it can’t go left (and vice-versa). This is because going left after a right would result in a U-bend that can’t be seen around.

An advantage over the other systems we tried was that this is all CPU-based and didn’t rely on expensive API functions like GetPixels() or raycasting to work, meaning we have greater control and flexibility over how it works.


Lighting Occlusion

Last time I talked about this there was a system to determine if a light should be on, using a render of the scene and use of special colours. We scrapped this in favour of a new simpler system. Lights aren’t actually that expensive in Unity, but shadows are, meaning if we can reduce the number of lights with shadows, we are 90% of the way there.

Effectively, the new light system fades light shadows over distance. The only trouble with this (and this is something we wanted to avoid) is that far away lights can bleed between rooms. Our solution to this is just position lights well so that they don’t. Additionally, there are some lights that end up casting large shadows, and these can be quite jarring when they fade. Fortunately, we can usually identify and workaround these on a case-by-case basis.


I hope you found something of this interesting! Until the next time!