Dynamic depth of field on the GPU – Part 1 of n

Modern 3D games use a bunch of tricks to convince our brains that we are viewing their world through some bizarre hybrid sense organ which consists of about 30% human eye and 70% movie camera. Hence we get lens flares, aperture changes and other movie staples which aren’t exactly true to life; we accept these effects probably because a) we are all so highly mediated these days that we expect the things which appear on our TVs/monitors to look like that and b) because they make shiny lights dance around the screen, and us primates love that stuff.

(An aside; anybody who wears glasses is totally used to lens flares, bloom lighting and film grain effects in everyday life, which is probably another reason why us nerds are so accepting of seeing the world as a movie. These settings can be temporarily toggled off with the use of a small amount of detergent and a soft cloth, but tend to return to the defaults over time).

What the human eye does have in spades, though, is dynamic depth of field. Anything outside of the centre of the field of view is out of focus and therefore appears blurred (and also in black and white, but let’s pretend we don’t know that). Humans generally focus on the thing in the centre of their visual field, even when the thing they are actually attending to isn’t (hence when you watch something out of the corner of your eye, it’s still blurry). Because depth of field effects weren’t at all viable on early graphics hardware, a lot of people have got used to everything in a scene having the same sharpness and dislike the addition of depth of field. However, used tastefully, it can nicely work as a framing effect; in addition it’s pretty handy to hide lower-resolution assets in the background.

The technique I am going to explain here has a major advantage for my purposes; the whole thing can be done as a post-process on the GPU, meaning that you don’t have to fiddle around with scene graphs or reading your depth buffer back for calculations on the CPU.

A static depth of field effect is easily added to a graphics pipeline; for example, you can set up a blur which uses your depth buffer such that the further an object is away, the more blurred it is. A dynamic depth of field is more complicated, and can be divided into several steps:

  1. Get some idea of the distance between the camera and the object being looked at. This can be approximated by sampling the depth at the centre of the screen. A method for getting linear depth from the depth buffer is needed.
  2. For each screen fragment, work out how much its depth differs from the focal depth.
  3. Perform a full-screen blur operation using this depth difference to determine the strength of the blur. This could, in ascending order of cost, be a box, gaussian or bokeh-style blur

This is easy to integrate into both forward and deferred rendering pipelines, and can be done entirely on the GPU as a post process using shaders. In addition, it serves as a useful demo of some handy shader techniques, such as how to linearise a depth buffer. The only real requirement is that you render your scene to a framebuffer object with an attached depth texture, rather than directly to the back buffer.

I will explain the steps required to produce the effect over a series of posts, each of which will last as long as my attention span does.

Step 1: Linear depth
For the purposes of this article, I’m going to assume that your main framebuffer object has an attached depth texture. You are going to need another framebuffer, the same dimensions as the main buffer, with a floating point texture attached, so that you can record the difference in depth of each fragment, compared with our estimated focal length. For simplicity, I have chosen to estimate the focal length and calculate the difference in the same shader.

When you draw anything in OpenGL using a depth buffer, the depth recorded isn’t linear. Instead, OpenGL uses most of its depth precision for the nearer objects in a scene and very little for further objects. This is very helpful in preventing ‘flimmering’ (the GL can’t decide which of two triangles is nearer, so it guesses; these guesses aren’t consistent each frame, and so the triangles flicker between the two different states. This looks terrible. Look for it in your favourite video game today!), but it means that you can’t use the depth buffer to meaningfully compare the distances to two points without a bit of work. Fortunately, we can produce a map of linearised depth, with all depths mapped to between 0.0 (near plane) and 1.0 (far plane):

//fragment shader code for calculating linear depth
 
//our main framebuffer's depth texture
uniform sampler2D depthTexture;
 
//near and far clipping planes for projection
uniform float near_clip;
uniform float far_clip;
 
//we are rendering a single quad
//which gives us texture coordinates from (0,0) to (1,1)
 
in vec2 texCoord;
 
out vec4 fragColor;
 
void main()
{
float z = texture(depthTexture, texCoord).x;
float z_n = 2.0 * z - 1.0;
float z_e = 2.0 * near_clip * far_clip / (far_clip + near_clip - z_n * (far_clip - near_clip));
float linear_z = z_e/far_clip;
 
fragColor = vec4(linear_z, linear_z, linear_z, 1.0);
}

For a clearer idea of what we’re actually producing, here’s shots of a linearised depth buffer, and a depth (not depth of field – blur is proportional to depth) blur effect generated.

Check the next article to see how to compare this depth with the focal depth.

Leave a Reply

Your email address will not be published. Required fields are marked *