Real-Time Per-Pixel Point Lights and Spot Lights in OpenGL
using nVidia Register Combiners
by Ronald Frazier ()

With the advancements in features and performance of modern consumer graphics cards, the quality and realism of real-time computer graphics (most notably games) has increased at an astonishing rate. The latest set of features (introduced with the nVidia GeForce series) includes hardware transform and lighting (hardware T&L) and per-pixel lighting. These features offer the potetial for 3D applications to provide even more realism in several ways. One of the ways to improve the realism of graphics is through improved dynamic lighting.

For several years, games have had to rely upon static pre-calculated lightmaps as the sole source of lighting. In recenet years dynamic lighting has begun to see its way into games in limited ways, but only as a small supplement to a predominantly lightmapped environment. One of the most popular ways to implement dynamic lighting in any API is through dynamic vertex lighting. While this can provide very realistic lighting under some circumstances, for performance reasons it has not enjoyed widespread implementation. This biggest problem facing vertex lighting is that, for relatively close up  lighting (ie: a flashlight, as opposed to sunlight), large surfaces must be tesselated to achieve any degree of realism. A flashlight shining on the very center of very large polygon will create a lighting effect that does not resemble a flashlight in any way. The only way to overcome this problem with vertex lighting is to tesselate the large polygon into several polygons that are relatively small compared to the light. The problem with this method is that is requires a lot of processing power from the CPU. With the advent of consumer graphics hardware featuring hardware transform and lighting, it becomes more reasonable to use highly tesselated surfaces under some circumstances, but other factors (including memory requirements and bus bandwidth limitations) prevent vertex lighting and high tesseleation from becoming the be-all/end-all to lighting problems. Even as the speed of CPUs and graphics cards increase to levels that make realistic vertex lighting feasible, there are still thousand of other ways in which this performance increase can be better utilized. Clearly another solution for dynamic lighting is needed. This solution is per-pixel lighting.

Per-Pixel lighting offers graphics applications the ability to provide extremely realistic lighting with a minimal of processing, bandwidth, and memory overhead. As we will see, per-pixel lighting offers the benefits of tesselating large polygons into several pixel-sized polygons without the performance impact of actually doing so. As we will see, per-pixel lighting also allows you to easily do things that are more complex using vertex lighting. While the techniques discussed here are equally applicaible to other 3D APIs (including Direct3D),  for this article I will focus solely on per-pixel lighting under OpenGL. Additionally, the techniques discussed here are only available on a few of the most modern graphics cards. By far, the most popular of these cards are the GeForce series of cards from nVidia, so this article will focus on these cards by using OpenGL extenstions that are currently proprietary to these cards.

nVidia Register Combiners
The driving force behind per-pixel lighting on GeForce cards is the presence of an extremly powerful and flexible processing engine known as the register combiners. In a nutshell, the register combiners allow you to implement a wide variety of calculations on the graphics cards on a per-pixel level, including signed addition, signed multiplication, and dot products. The operations can be performed on a wide varitey of operands including texture fragment colors, polygon colors, and even the results of the calculations performed by other combiners. The complete discussion of register combiners is beyond the scope of this article. If you are not already familiar with the features and functionality of the nVidia register combiner, please read one of the many documents on the subject available in the developers section of the nVidia web site (http://www.nvidia.com/developer).

Per-Pixel Point Lighting
One of the simplest and most common uses of lighting are point lights. A point light is any type of light that originates from a distinguishable source and radiates equally in all directions (not taking into account the effects of absorption and reflection from other objects). The most common type of point light in everyday life is a light bulb. The two important properties of point lights are that they radiate equally in all directions and that they attenuate over distance. The theory of implementing point lights is discussed in a presentation by Sim Dietrich at the 2000 Game Developers Conference titled Per-Pixel Lighting. A copy of this presentation is currently avialable on the nVidia developers site.

Calculating Attenuation
To make per-pixel lighting work, we need a way to represent light intensity and attenuation over a distance d. Since the values that the register combiners operate on must lie in the -1 to 1 range, we need a way to provide or calculate some value that lies within this range to represent the intensity and attenuation. One way to do this is to think of light intensity ranging from 0 (no light) to1 (full light). We can then can then calculate light intensity as:

                    Intensity = 1 - Attenuation

We then need to define Attenuation so that is has a value of 0 at distance 0 and a value of 1 at distance R (where R is the radius of the light). To do this, we will use the formula:

                    Attenuation = d² / R²

The only problem with this formula is that Attenuation approaches infinity as the distance d approaches infinity. Since values greater than 1 are out of range of the register combiners, and since it doesnt make sense to attenuate light more than 100%, we will clamp the result of this calculation to the 0 to 1 range. Combining these equations, we come up with:

                    Intensity = 1 - d² / R²

Substituting in the equation for calculating distance d from x, y, and z, we get the final equation:

                    Intensity = 1 - (x² + y² + z²)/R²              or         Intensity = 1 - (x/R)² - (y/R)² - (z/R)²

Now we need some ways to calculate this formula inside of the register combiners. The way we will do this is to break this formula into parts. If we can store the value (x/R)² + (y/R)² in texure unit 0,  store the value (z/R)² in texture unit one, and map the x,y,z coordinates (relative to the point light)  to texture coordinates properly, we can then calculate intensity for a given coordinate as:

                    Intensity = 1 - Texture0 - Texture 1            or         Intensity = 1 - (Texture0 + Texture 1)

How do we store these values into textures? For texture 0, we need a 2D texture that maps coordinates (x,y) to a single value x² + y². For texture 1, we need a 1D texture that maps the coordinate z to the value z². The following textures will do this for us.

2D Texture

1D Texture (stretched vertically for clarity)

Remember that we want to clamp values to the 0 to 1 range, so these textures should be created with the wrap mode set to CLAMP or CLAMP_TO_EDGE. Now, using the above textures, we need to map the (x,y,z) coordinates into this texture. To do this we can calculate the distance from the light as:

                    x₀ = (x - lightX) / R
                    y₀ = (y - lightY) / R
                    z₀ = (z - lightZ) / R

Note that we scaled each distance by dividing it by the light radius R. This allows us to ensure that distances from -R to R lie in the -1 to 1 range. The next step is to map these (x₀, y₀, z₀) distance which lie in the -1 to 1 range into (s, t, r) texture coordinates which lie in the 0 to 1 range. To do so we calculate:

                    s = x₀/2 + 0.5
                    t = y₀/2 + 0.5
                    r = z₀/2 + 0.5

Then, we can use coordinates (s,t) as the texture coordinates for the 2D Texture 0, and use the (r) coordinate as the texture coordinate for the 1D Texture 1. Finally, now that we have the Intensity of the light calculated, we can multiply this by the color of the light to get the distance attenuated light value for the current pixel.

Configuring the Register Combiners for Per-Pixel Point Lighting
Now that we know how to generate the light color, all that remains is to configure the register combiners to perform the calculations for us. The following code will accomplish this:

Rendering the Point Light
Finally, given the (x,y,z) and the derived (s,t,r) calculated as shown above, we can now feed these into the hardware using the following code:

Now, all we need to do is multiply the resulting value by the base texture color for the pixel to acheive the final textured lighted pixel. To do this, we need to draw the base texture in one pass and then generate the light color by re-rendering the geometry using the above code and the appropriate blend mode. The result is a per-pixel lighted point light. Below are a few screen shots demonstrating this technique.

Sample Images

Per-Pixel Spot Lighting
Using the above point lighting as a basis it would be nice if we could expand upon it to create a spot light. A spot light is similar to a point light in that it has a definited position and it attenuates over distance. However, unlike a point light, a spot light does not radiate equally in all directions. Instead, a spot light typically has a conical frustum in which it shines. This frustum is defined by the direction of the center of the frustum, and the angle between the center and outer edge of the frustum. Depending on the design of the spot light, the light may also attenuate as the angle from the center of the frustum increases, or it may maintain full strength from the center to the outer edges of the frustum. The best example of an everyday spot light is a flashlight.

Converting a Point Light Into a Spot Light
There are a probably at least a few ways to create a spot light. Working from the point light as our basis, what we would ideally like to do is to create a texture that defines a circular area which we could project out from the center of the point light to create the conical frustum. This texture could contain white or grey inside the frustum, and black outside the frustum. We could then use the per-pixel color of this projected texture and multiply it by the distance attenuated point light color to get our final color. The result would be that pixels outside the spot light frustum would be multiplied by zero and thus filtered out. While this should work under ordinary circumstances, there are some cases where you can run into problems. One instance is if you tried to make a spot light that extended 90° from the center (ie: like a point light, but it only shines forward and not backward). Additionally trying to make a spot light that radiates more than 90° from the center would be difficult (to say the least). It would be nice if we could find a way to "project" a texture out across the entire 360° of a sphere . . . Environment Mapping to the rescue.

Using environment mapping, we could create an environment map the filters out any portion of the spherical point light that we desired. The next question is what type of environment mapping to use. While any of the environment mapping techniques would do with enough work, the simplest and most straight forward environment mapping technique to use is the Cubic Environment Map. While this technique is rather new and not supported by most graphics cards, it is luckily on the feature list of the GeForece card.

So now, what we want to do is to calculate the point light as normal, and then, for each vertex, calculate the direction from the light center to the vertex and use this as a lookup into our cubemap. There is only one problem with this idea. The GeForce card (along with almost every other grahphics card in existence) only supports 2 simultaneous texture, both of which we are using for the pointlight. This leaves us with 2 options. We can either attempt to modify the point light to require only one texture unit, or we could create the spot light in 2 separate rendering passes. Expanding the rendering to 2 passes seems like the most obvious approach. However, if we do this, it will make rendering spot lights slower. Additionally, later on, when we explore combining multiple lights into the same scene, we would begin to see some artifacts in some situations where 2 or more lights shine on the same pixel. In some cases, we could arbitrarily say that our scene would be limited to one light, so we could accept this. However, for the general case, we need to generate the lighting values in a single pass. The only option left is to simplify the point light equation down to only require a single texture unit, leaving the remaining texture unit free for the Cubic Environment Map.

Simplifying a Point Light to Use Only One Texture
The question now becomes how to simplify the pointlight to one texture. If we examine the equations carefully, we can see a special case where the pointlight equation only requires a single texture. If the plane of the polygon lies parallel to the Z=0 plane, then we can continue to calculate the x and y attenuation using the 2D texture. However, note that regardless of where the plane lies and where the point light lies, the z distance from the point light to the plane will be the same across the entire plane. Therefore, instead of looking into a 1D texture to determine the z² distance attenuation, we can just calculate the distance from the point light to the polygon plane and store this into one of the constant color registers. The figure below  shows an example of this special case for clarity.

Looking from overhead of the point light

Now, how can we apply this special case to any general polygon? Rember that for right now we are only trying to calculate radial attenuation for a point light (this part has nothing to do with a spot light). Knowning that a point light radiates equally in all directions, we can concluded that if we rotate a plane to be parallel to the z=0 plane, we can simplify any polygon down to the special case noted above. So now we need to know how to rotate the polygon. To do this we need to determin what axis to rotate about, and how many degrees to actually rotate. Knowing the plane normal N for the polygon and the negative z vector NZ, we can calculate the axis of rotation as the cross product of these vectors (cross product calculation denoted by x):

                        rotationAxis = N x NZ

This will give us the axis of rotation for the plane. Next we need to calculate the angle of rotation. Using some trigonometry, we can use the inverse-sin and inverse-cos functions as well as a dot product operation (denoted by dotProduct) to determine the angle:

                        rotationAngle = asinf(length(rotationAxis)) * 180.0f/3.14159f;
                        if (acosf(N dotProduct NZ) <= (3.14159f/2.0f))
                                rotationAngle = 180 -rotationAngle;

If the above calculation doesnt make any sense, you might want to break out a good trig or calculus book. None the less, take my word that it works. Now, using the rotated plane, we can use calculate the (s, t) texture coordinates just as we did in the normal point light. To get the z distance, we just need to calculate the distance from the point light to the unrotated plane using the following calculation:

                        ZDistance = A*x + B*y + C*z + D

where (A,B,C,D) are the coefficients of the plane equation for the polygon and (x,y,z) is the location of the point light. Now, using the (s,t) texture coordinates, and using ZDistance² in place of the 1D texture, we can calculate the radial attenuations using only a constant value and a singe texture. There is one tradeoff to this technique, and that is that we need to prepare the light before drawing each individual plane. This does limit our ability to batch multiple polygons together and submit them to be rendered at once. However, we can still batch polygons that have the same plane equation. The following code (extracted from the CSpotlightTexture class in the accompanying source code) demonstrates the prepareLightForPlane function. Note that it makes use of a CVector class and a CPlane class also provided in the accompanying source code.

Note that in the after determining the rotation axis and angle, we put these results into the texture matrix. This gives us the advantage of not manually calculating the rotation for each vertex. Additionally, note that the other operations performed in the regular point light have also been moved into the texture matrix. This way, we can push as much calculation as possible off onto the hardware T&L available on the GeForce card.

Now, at long last, we have simplified the point light down to using a single texture unit. Before proceeding further, we should note a few things. With the availablity of another texture unit, we could maximize performance of a standard point light in a few ways. The first is that we could combine the light calculation rendering and the base texture rendering into a single pass. Note, however, that this will still give us the same problems when combining multiple light sources that was mentioned earlier. The other option is that we might be able (with enough manipulation) to calculate the combined lighting for 2 separate point lights simultaneously. However, I have not attempted this or thought it through fully, so it may not be possible after all, but is worthy of future investigation. It is also important to note that by simplifying the point light to use a single texture, we would again lose the ability to batch process lots of polygons together, as we would need to set up the point light for each individual plane equation. Whether or no this tradoff is acceptable would depend on the individual application.

Filtering a Point Light into a Spot Light
Ok, since we are done with the pointlight equation, we can now move on to applying the Cubic Environment Map to convert the point light into a spot light. If we create a cube map with all black sides, and a white circle on the negative Z axis, we can then multiply the point light color by the cube map value (determined with the vector from the light to the vertex) to come up with the filtered color value. The following image shows the 6 sides of the cube map labeled in red:

Spot Light Cube Map

Now, once again, we can use the vector from the light to the vertex to perform a lookup into the cubemap. However, there is one more point we need to consider. If we perform this as is, then we will always see the light facing down the negative Z axis. Ideallay, we would like to be able to rotate the light in any direction. In order to do this, we either need to store or calculate the rotational angles of the light directions. This can get kind of messy as rotations about various axis accumulate. Ideally, we would like to just store the direction that the light is facing in and use this as the basis for our calculations. We could try to use cross products, dot products, inverse sin and inverse cos to calculate the rotational matrix each frame, but this is more time consuming than we would like. Linear algebra to the rescue!

If we store an ortho-normal basis for the light (recall that a 3D ortho-normal basis consists of 3 perpendicular vectors that define a coordinate system) consisting of the forward direction, the up direction, and the right direction, we can very easily use some of the principles of viewing transformations to quickly and efficiently build a matrix to rotate the cubemap into the desired orientation. And the best part is that by using ONBs, we can build this matrix without any mathematical calculations. Things just keep getting better, too, because if we feed this matrix into the texture matrix, we can let the hardware T&L do all the work of multiplying each vertex by this matrix for us. The following code fragment sets up the texture matrix to rotate the cube map. Note that this code makes use of a COrthoNormalBasis class which is, as usual, provided in the accompanying source code.

One thing to note in the above bit of code is that we scaled the x and y texture coordinates. The reason we do this is so that we can easily adjust the angle of frustum dynamically from 0° to 90° (representing a sphere chopped in half) without have to generate and use multiple cube maps. By default, the unscaled cubemap shown above will give us a 45° frustum angle.

Configuring the Register Combiners for Per-Pixel Spot Lighting
Now that we see how to setup both the point light component and the spot light filtering component, we just need to know how to setup the register combiners to perform the desired operation. As before, texture 0 is the 2D radial component of the point light, and as shown above, we put the z attenuation into the Constant Color 0 register. Now we set up the first register combiner to calculate the combined radial attenuation by adding these 2 components together and storing the result in the spare 0 register. Then we configure the final combiner to calculate Light Color * Texture 1 Cube Map * (1- attenuation) and we have the final resulting filtered spot light. Here is the code for setting up the register combiners.

Rendering the Spot Light
Finally, given the (x,y,z) coordinate of each vertex, we can now feed these into the hardware using the following code:

And thats all there is to the spot light. Now to please your eyes, here are some images of the spot light

Neat Tricks Using the Spot Light
One of the advantages of using a cube map instead of a projected texture to generate the spot light is that we can perform some interesting tricks just by using various cube maps. Consider the following cube maps

Disco Light Cube Map

Beacon Cube Map

Using the above cubemaps we can generate a disco light or a police beacon which we can rotate to produce some nice visual effects. Here are some shots of it in action:

Multiple Lights in a Scene
One last consideration we need to make is when we have more than one light illuminating the sceen. To avoid artifacts in overlapping lights, we need to take special steps when rendering. The first thing to do is to clear the back buffer to the ambiant light color. Then, additively render each light into the scene to create a lightmap for the entire scene. Note that in this step we can even safely combine point lights and spot lights together. Finally, after all lights are rendered, render the base textures using dst_color : zero (ed: I originally said src_color : zero by mistake) blending so that the base texture color gets multiplied by the light color producing the final output. The images below demonstrate this.

Improvements
Like everything else, there are ways to improve upon these per-pixel lights. One option is to incorporate some type of bump mapping. This would force the light into requiring multiple passes, but it would improve the realism of the scene. Another option would be to encorporate gloss maps and specular highlights. Finally, it should be noted that this lighting algorithm does not take into account the effects of occluding geometry. Therefore this algorithm will not produce any type of shadows. To implement shadows, you could combine this technique with either stencil shadowing or depth buffer shadow casting. Information on all of these techniques are available on the nVidia developers site.

Conclusion
Per-pixel lighting provides many unique opportunities for realistic 3D rendering. We should all look forward to the day when all graphics hardware supports these advanced features. Finally, I would like to throw out a big thumbs up to nVidia for being the company to continually push the envelope when it comes to incorporating the most advanced technology and features into consumer graphics cards.

Source Code and Executable
The source code, compiled executable, and necessary image files for the demo application can be downloaded here.