• Article

October 2014

Volume 29 Number 10

DirectX Factor : Pixel Shaders and the Reflection of Light

Charles Petzold

Charles PetzoldIf you could see photons … well, you can see photons, or at least some of them. Photons are the particles that make up electromagnetic radiation, and the eye is sensitive to photons with wavelengths within the range of visible light.

But you can’t see photons as they fly about all over the place. That would surely be interesting. Sometimes photons go right through objects; sometimes they’re absorbed; sometimes they’re reflected; but often it’s a combination of all of these effects. Some of the photons that bounce off objects eventually reach your eyes, giving each object its particular color and texture.

For extremely high-quality 3D graphics, a technique called ray tracing can actually plot out a simulation of the paths of these myriad photons to mimic the effects of reflection and shadows. But much simpler techniques are available for more conventional needs. This is often the case when using Direct3D—or, in my case, writing custom effects in Direct2D that make use of 3D.

Reusing the Effect

As you’ve seen in previous installments of this column, a Direct2D effect is basically a wrapper for code that runs on the GPU. Such code is known as a shader, and the most important are the vertex shader and the pixel shader. The code in each of these shaders is called at the video refresh rate of the display. The vertex shader is called for each of the three vertices in each triangle that make up the graphical objects displayed by the effect, while the pixel shader is called for every pixel within these triangles.

Obviously, the pixel shader is called much more frequently than the vertex shader, so it makes sense to keep as much processing as possible in the vertex shader rather than the pixel shader. This isn’t always possible, however, and when using these shaders to simulate the reflection of light, it’s usually the balance and interaction between these two shaders that governs the sophistication and flexibility of the shading.

In the August issue of this magazine, I presented a Direct2D effect called RotatingTriangleEffect that constructed a vertex buffer consisting of points and colors, and allowed applying standard model and camera transforms to the vertices. I used this effect for rotating three triangles. That’s not a lot of data. Three triangles involve just a total of nine vertices, and I mentioned at the time that the same effect could be used for a much larger vertex buffer.

Let’s try it out: The downloadable program (msdn.microsoft.com/magazine/msdnmag1014) for this column is called ShadedCircularText, and it uses RotatingTriangleEffect without a single change.

The ShadedCircularText program returns to the problem I began exploring earlier this year of displaying tessellated 2D text in three dimensions. The constructor of the ShadedCircularTextRenderer class loads in a font file, creates a font face from it, and then calls GetGlyphRunOutline to obtain a path geometry of the character outlines. This path geometry is then tessellated using a class I created called InterrogableTessellationSink that accumulates the actual triangles.

After registering RotatingTriangleEffect, ShadedCircularText­Renderer creates an ID2D1Effect object based on this effect. It then converts the triangles of the tessellated text into vertices on the surface of a sphere, basically wrapping the text around the equator and bending it toward the poles. The color of each vertex is based on a hue derived from the X coordinate of the original text geometry. This creates a rainbow-like effect, and Figure 1 shows the result.

A 3D Text Rainbow from ShadedCircularText
Figure 1 A 3D Text Rainbow from ShadedCircularText

As you can see, a little menu adorns the top. The program actually incorporates three additional Direct2D effects that implement more traditional shading models. All of them use the same points, the same transforms, and the same animations, so you can switch between them to see the difference. The differences involve only the color shading of the triangles.

The bottom-right corner has a display of performance in frames-per-second, but you’ll discover that nothing in this program causes that to drop much below 60 except if something else is going on.

Gouraud Shading

As photons are flying all around us, they often bounce off nitrogen and oxygen molecules in the air. Even on an overcast day with no direct sunlight, there’s still a considerable amount of ambient light. Ambient light tends to illuminate objects in a very uniform manner.

Perhaps you have an object that’s a greenish blue, with an RGB value of (0, 0.5, 1.0). If the ambient light is one-quarter of full intensity white, you can assign an RGB value to the light of (0.25, 0.25, 0.25). The perceived color of this object is the product of the red, green, and blue components of these numbers, or (0, 0.125, 0.25). It’s still a greenish blue, but much darker.

But simple 3D scenes don’t live by ambient light alone. In real life, objects normally have a lot of color variation on their surfaces, so even if they’re uniformly illuminated, the objects still have visible textures. But in a simple 3D scene, a greenish-blue object illuminated only by ambient light will merely look like an undifferentiated slab of uniform color.

For this reason, simple 3D scenes benefit enormously from some directional light. It’s simplest to assume that this light comes from a far distance (like the sun), so the direction of the light is just a single vector that applies to the entire scene. If there’s only one light source, generally it’s assumed to come from over the viewer’s left shoulder, so perhaps the vector is (1, -1, -1) in a right-hand coordinate system. This directional light also has a color, perhaps (0.75, 0.75, 0.75), so together with the ambient light of (0.25, 0.25, 0.25), maximum illumination is sometimes achieved.

The amount of directional light a surface reflects depends on the angle the light makes with the surface. (This is a concept explored in my May 2014 DirectX Factor column.) The maximum reflection occurs when the directional light is perpendicular to the surface, and reflected light decreases to zero when the light is tangent to the surface or coming from somewhere behind the surface.

The Lambert Cosine Law—named after German mathematician and physicist Johann Heinrich Lambert (1728 – 1777)—says that the fraction of light reflected from a surface is the negative cosine of the angle between the direction of the light, and the direction of a vector perpendicular to the surface, which is called a surface normal. If these two vectors are normalized—that is, if they have a magnitude of 1—this cosine of the angle between the two vectors is the same as the dot product of the vectors.

For example, if the light strikes a particular surface at a 45-degree angle, the cosine is approximately 0.7, so multiply that by the direc­tional light color of (0.75, 0.75, 0.75) and the color of the object (0, 0.5, 1.0) to derive the color of the object from directional light of (0, 0.26, 0.53). Add that to the color from ambient light.

However, keep in mind that curved objects in 3D scenes aren’t actually curved. Everything in the scene consists of flat triangles. If the illumination of each triangle is based on a surface normal perpendicular to the triangle itself, each triangle will have a different uniform color. This is fine for the Platonic solids such as those displayed in my May 2014 column, but not so good for curved surfaces. For curved surfaces, you want the colors of the triangles to blend with each other.

This means it’s necessary for each triangle to have a graduated color rather than a uniform color. The color from directional light can’t be based on a single surface normal for the triangle. Instead, each vertex of the triangle should have a different color based on the surface normal at that vertex. These vertex colors can then be interpolated over all the pixels of the triangle. Adjacent triangles then blend in with each other to resemble a curved surface.

This type of shading was invented by French computer scientist Henri Gouraud (b. 1944) in a paper published in 1971, and is therefore known as Gouraud shading.

Gouraud shading is the second option implemented in the ShadedCircularText program. The effect itself is called GouraudShadingEffect, and it requires a vertex buffer with somewhat more data:

struct PositionNormalColorVertex
{
  DirectX::XMFLOAT3 position;
  DirectX::XMFLOAT3 normal;
  DirectX::XMFLOAT3 color;
  DirectX::XMFLOAT3 backColor;
};

Interestingly, because the text is effectively being wrapped around a sphere centered at the point (0, 0, 0), the surface normal at each vertex is the same as the position, but normalized to have a magnitude of 1. The effect allows unique colors for every vertex, but in this program every vertex gets the same color, which is (0, 0.5, 1) and the same backColor of (0.5, 0.5, 0.5), which is the color to be used if the back of a surface faces the viewer.

The GouraudShadingEffect also requires more effect properties. It must be possible to set the ambient light color, directional light color, and the vector direction of the directional light. The GouraudShadingEffect transfers all these values to a larger constant buffer for the vertex shader. The vertex shader itself is shown in Figure 2.

Figure 2 The Vertex Shader for Gouraud Shading

// Per-vertex data input to the vertex shader
struct VertexShaderInput
{
  float3 position : MESH_POSITION;
  float3 normal : NORMAL;
  float3 color : COLOR0;
  float3 backColor : COLOR1;
};
// Per-vertex data output from the vertex shader
struct VertexShaderOutput
{
  float4 clipSpaceOutput : SV_POSITION;
  float4 sceneSpaceOutput : SCENE_POSITION;
  float3 color : COLOR0;
};
// Constant buffer provided by effect.
cbuffer VertexShaderConstantBuffer : register(b1)
{
  float4x4 modelMatrix;
  float4x4 viewMatrix;
  float4x4 projectionMatrix;
  float4 ambientLight;
  float4 directionalLight;
  float4 lightDirection;
};
// Called for each vertex.
VertexShaderOutput main(VertexShaderInput input)
{
  // Output structure
  VertexShaderOutput output;
  // Get the input vertex, and include a W coordinate
  float4 pos = float4(input.position.xyz, 1.0f);
  // Pass through the resultant scene space output value
  //  (not necessary -- can be removed from both shaders)
  output.sceneSpaceOutput = pos;
  // Apply transforms to that vertex
  pos = mul(pos, modelMatrix);
  pos = mul(pos, viewMatrix);
  pos = mul(pos, projectionMatrix);
  // The result is clip space output
  output.clipSpaceOutput = pos;
  // Apply model transform to normal
  float4 normal = float4(input.normal, 0);
  normal = mul(normal, modelMatrix);
  // Find angle between light and normal
  float3 lightDir = normalize(lightDirection.xyz);
  float cosine = -dot(normal.xyz, lightDir);
  cosine = max(cosine, 0);
  // Apply view transform to normal
  normal = mul(normal, viewMatrix);
  // Check if normal pointing at viewer
  if (normal.z > 0)
  {
    output.color = (ambientLight.xyz + cosine *
                    directionalLight.xyz) * input.color;
  }
  else
  {
    output.color = input.backColor;
  }
  return output;
}

The pixel shader is the same as for the RotatingTriangleEffect, and is shown in Figure 3. The interpolation of the vertex colors over the entire triangle occurs behind the scenes between the vertex shader and the pixel shader, so the pixel shader simply passes the color on to be displayed.

Figure 3 The Pixel Shader for Gouraud Shading

// Per-pixel data input to the pixel shader
struct PixelShaderInput
{
  float4 clipSpaceOutput : SV_POSITION;
  float4 sceneSpaceOutput : SCENE_POSITION;
  float3 color : COLOR0;
};
// Called for each pixel
float4 main(PixelShaderInput input) : SV_TARGET
{
  // Simply return color with opacity of 1
  return float4(input.color, 1);
}

The result is shown in Figure 4, this time on Windows Phone 8.1 rather than Windows 8.1. The ShadedCircularText solution was created in Visual Studio with the new Universal App template and can be compiled for either platform. All code is shared between the two platforms except for the App and DirectXPage classes. The difference in the layouts of the two programs suggests why having different page definitions is often a good idea, even if the functionality of the program is fundamentally the same.

The Display of the Gouraud Shading Model
Figure 4 The Display of the Gouraud Shading Model

As you can see, the figure is lighter in its upper-left area, clearly showing the effect of directional light and aiding in the illusion of a rounded appearance to the surface.

The Phong Improvements

Gouraud shading is a time-honored technique, but it has a fundamental flaw: In Gouraud shading, the amount of directional light reflected in the center of a triangle is an interpolated value of the light reflected at the vertices. The light reflected at the vertices is based on the cosine of the angle between the light direction and the surface normals at those vertices.

But the light reflected in the center of the triangle should really be based on the surface normal at that point. In other words, the colors shouldn’t be interpolated over the triangle; instead, the surface normals should be interpolated over the triangle’s surface, and the reflected light calculated for each pixel based on that normal.

Enter Vietnamese-born computer scientist Pui Tuong Phong (1942-1975), who died of leukemia at the age of 32. In his 1973 doctoral dissertation, Phong described a somewhat different shading algorithm. Rather than interpolate vertex colors over the triangle, the vertex normals are interpolated over the triangle, and then reflected light is calculated from those.

In a practical sense, Phong shading requires the calculation of reflected light to be moved from the vertex shader to the pixel shader, along with the section of the constant buffer devoted to that job. This increases the amount of per-pixel processing immensely, but, fortunately, it’s being done on the GPU where you hope it won’t seem to make much of a difference.

The vertex shader for the Phong shading model is shown in Figure 5. Some of the input data—such as the color and back color—are simply passed on the pixel shader. But it’s still useful to apply all the transforms here. The world transform and both camera transforms must be applied to the positions, while two normals are also calculated—one with only the model transform for the reflected light, and another with the view transform to determine whether a surface faces toward or away from the viewer.

Figure 5 The Vertex Shader for the Phong Shading Model

// Per-vertex data input to the vertex shader
struct VertexShaderInput
{
  float3 position : MESH_POSITION;
  float3 normal : NORMAL;
  float3 color : COLOR0;
  float3 backColor : COLOR1;
};
// Per-vertex data output from the vertex shader
struct VertexShaderOutput
{
  float4 clipSpaceOutput : SV_POSITION;
  float4 sceneSpaceOutput : SCENE_POSITION;
  float3 normalModel : NORMAL0;
  float3 normalView : NORMAL1;
  float3 color : COLOR0;
  float3 backColor : COLOR1;
};
// Constant buffer provided by effect
cbuffer VertexShaderConstantBuffer : register(b1)
{
  float4x4 modelMatrix;
  float4x4 viewMatrix;
  float4x4 projectionMatrix;
};
// Called for each vertex
VertexShaderOutput main(VertexShaderInput input)
{
  // Output structure
  VertexShaderOutput output;
  // Get the input vertex, and include a W coordinate
  float4 pos = float4(input.position.xyz, 1.0f);
  // Pass through the resultant scene space output value
  // (not necessary — can be removed from both shaders)
  output.sceneSpaceOutput = pos;
  // Apply transforms to that vertex
  pos = mul(pos, modelMatrix);
  pos = mul(pos, viewMatrix);
  pos = mul(pos, projectionMatrix);
  // The result is clip space output
  output.clipSpaceOutput = pos;
  // Apply model transform to normal
  float4 normal = float4(input.normal, 0);
  normal = mul(normal, modelMatrix);
  output.normalModel = normal.xyz;
  // Apply view transform to normal
  normal = mul(normal, viewMatrix);
  output.normalView = normal.xyz;
  // Transfer colors
  output.color = input.color;
  output.backColor = input.backColor;
  return output;
}

As the output from the vertex shader becomes input to the pixel shader, these normals are interpolated over the triangle’s surface. The pixel shader can then finish up the job by calculating the reflected light, as shown in Figure 6.

Figure 6 The Pixel Shader for the Phong Shading Model

// Per-pixel data input to the pixel shader
struct PixelShaderInput
{
  float4 clipSpaceOutput : SV_POSITION;
  float4 sceneSpaceOutput : SCENE_POSITION;
  float3 normalModel : NORMAL0;
  float3 normalView : NORMAL1;
  float3 color : COLOR0;
  float3 backColor : COLOR1;
};
// Constant buffer provided by effect
cbuffer PixelShaderConstantBuffer : register(b0)
{
  float4 ambientLight;
  float4 directionalLight;
  float4 lightDirection;
};
// Called for each pixel
float4 main(PixelShaderInput input) : SV_TARGET
{
  // Find angle between light and normal
  float3 lightDir = normalize(lightDirection.xyz);
  float cosine = -dot(input.normalModel, lightDir);
  cosine = max(cosine, 0);
  float3 color;
  // Check if normal pointing at viewer
  if (input.normalView.z > 0)
  {
    color = (ambientLight.xyz + cosine *
      directionalLight.xyz) * input.color;
  }
  else
  {
    color = input.backColor;
  }
  // Return color with opacity of 1
  return float4(color, 1);
}

However, I’m not going to show you a screenshot of the result. It’s pretty much visually identical to the Gouraud shading. Gouraud shading really is a good approximation.

Specular Highlights

The real importance of Phong shading is that it makes possible other features that rely upon a more accurate surface normal.

So far in this article, you’ve seen shading that’s appropriate for diffuse surfaces. These are surfaces that are rather rough and dull and that tend to scatter light reflected off their surfaces.

A surface that’s somewhat glossy reflects light a little differently. If a surface is tilted just so, directional light could bounce off and go straight to the viewer’s eye. This is usually perceived as bright white light, and it’s known as a specular highlight. You can see the rather exaggerated effect in Figure 7. If the figure had sharper curves, the white light would be more localized.

The Specular Highlight Display
Figure 7 The Specular Highlight Display

Getting this effect seems at first as if it might be computationally complex, but it’s only a few lines of code in the pixel shader. This particular technique was developed by NASA graphics maven Jim Blinn (b. 1949).

We first need a vector indicating the direction that the viewer of the 3D scene is looking. This is very easy because the view camera transform has adjusted all coordinates so the viewer is looking straight down the Z axis:

float3 viewVector = float3(0, 0, -1);

Next, calculate a vector that’s halfway between that view vector and the light direction:

float3 halfway = -normalize(viewVector + lightDirection.xyz);

Notice the negative sign. This makes the vector point in the opposite direction—midway between the source of the light and the viewer.

If a particular triangle contains a surface normal that corresponds exactly with this halfway vector, it means that light is bouncing off the surface directly into the viewer’s eye. This results in maximum specular highlighting.

Lesser highlighting results from non-zero angles between the halfway vector and the surface normal. This is another application for the cosine between the two vectors, which is the same as the dot product if the two vectors are normalized:

float dotProduct = max(0.0f, dot(input.normalView, halfway));

This value of dotProduct ranges from 1 for maximum specular highlighting when the angle between the two vectors is 0, to 0 for no specular highlighting, which occurs when the two vectors are perpendicular.

However, specular highlighting shouldn’t be visible for all angles between 0 and 90 degrees. It should be localized. It should exist only for very small angles between those two vectors. You need a function that won’t affect a dot product of 1, but will cause values less than 1 to become much lower. This is the pow function:

float specularLuminance = pow(dotProduct, 20);

This pow function takes the dot product to the 20th power. If the dot product is 1, the pow function returns 1. If the dot product is 0.7 (which results from an angle of 45 degrees between the two vectors), then the pow function returns 0.0008, which is effectively 0 as far as lighting goes. Use higher exponent values to make the effect even more localized.

Now all that’s necessary is to multiply this factor by the directional light color and add it to the color already calculated from ambient light and directional light:

color += specularLuminance * directionalLight.xyz;

That creates a splash of white light as the animation turns the figure.

Farewell

And with that, the DirectX Factor column comes to a close. This plunge into DirectX has been one of the most challenging jobs of my career, but consequently also one of the most rewarding, and I hope to have an opportunity someday to return to this powerful technology.

Charles Petzold is a longtime contributor to MSDN Magazine and the author of “Programming Windows, 6th Edition” (Microsoft Press, 2013), a book about writing applications for Windows 8. His Web site is charlespetzold.com.

Thanks to the following Microsoft technical expert for reviewing this article: Doug Erickson

This issue marks Charles Petzold’s last as a regular columnist in MSDN Magazine. Charles is leaving to join the team at Xamarin, a leading provider of cross-platform tools leveraging the Microsoft .NET Framework. Charles has been associated with MSDN Magazine for decades and has authored numerous regular columns, including Foundations, UI Frontiers and DirectX Factor. We wish him well on his new endeavors.