April 15, 2010

Nontechnical Explanation of Color in 3D Games

Actual pixel value sent to the pixel on the monitor is, in a typical 3D game:

Texture value * Light value, where each value is a scale from 0.0 to 1.0.

The scale measures from black to white. Or rather, from the monitor displaying that pixel as dark as it go at 0 to as bright as it can go at 1. Different monitors have different maximum outputs, but because of how our eyes work we don't actually notice this as anything except "better contrast". The fact that the colors are recorded as relative to the amount the monitor can output is the cause of problems when designers are making art to be displayed on different monitors, or for print, or whatever.

(As opposed to a 2D game which typically merely blits, i.e. copies the values directly from a sprite image, which is actually analogous to a texture, in a way.)

Thus a 'fully lit' scene will display exactly what the texture is (texture value * 1), while a 'fully dark' scene will display black (texture value * 0). A black texture will always show up as black, and a white texture will show up as whatever the light hitting it is.

Texture value is the value of the pixel of the texture map at that position on the screen, based on the position of the polygon it's mapped to, the way it's mapped to the poly, occlusion, and so on. This is all going on in the GPU is is the majority of the math needed for rendering. You've got to worry about how the poly is rotated, scaled, skewed, and so on. But basically eventually you get a result that says, "This pixel is pulling from THIS spot on the texture image." So this is what I mean by it being analogous to blitting: You've got an image, and you want to get a pixel it from it and draw it at the right spot on the screen.

Light value is the amount of light hitting that spot on that poly from lights placed in the scene. Each light object has it's own value from 0 (no light) to 1 (max light) Note I don't say 'black' and 'white', because that's technically not what's happening.

The lights are all casting some amount of their value on that pixel, usually based on distance. In nature, this falloff has to do with the brightness of the light and the inverse-square law: The intensity of illumination is proportional to the inverse square of the distance from the light source.

This law doesn't actually work though because the value of the light is NOT the luminosity, which I'll explain later.

Rather, the typical game physically sets a radius for each light (set by the level designer, actually) and makes it so the light falls off linearly (although often it's a square falloff) until at the max radius, it's 0. So at the center of the sphere, the light amount if equal to whatever the light is set to, let's say 1,0.5,0.5 for a pinkish light, and at the surface of the sphere it's 0,0,0.

So, the game is finding how much each light in the scene is casting on our pixel, based on each light's distance and value. Once we have a list of all these values, simply add them all together! This part is easy, and even naturally results in lifelike color blending. A red 1,0,0 light and a blue 0,0,1 light will combine to end up with a purple 1,0,1 result. This is why colored lighting was a very early bullet point feature of 3D games.

Now, whatever value you have is the total amount of light being cast on that pixel, which can be anywhere from 0 to infinity (assuming you can place anywhere from 0 to infinite lights). We can't actually store a value on that scale though, we want a scale from 0 to 1. You could set 1 to be whatever the highest value in the scene is and normalize everything else based on that, OR you could simply clamp at at 1,1,1. This is what most games do. Thus, in the level designer's mind, lights will never add together so much that they make the texture render brighter than it is.

Thus if you take a pixel, and its got 0.5 light coming from light A, and 0.75 light coming from light B (after distance check), they'll add together to 1.25 light and be clamped to 1, and we say that that pixel is "fully lit" at 1. In practice it might even be rare for a pixel to ever be fully lit though. It's usually some distance away from all the lights in the scene, and they're often not even max value lights. This is why in, say, Quake 2 if you disable lighting and let everything just be based on texture value everything looks really bright and stark. You're not used to ever seeing those values.

But basically nothing will ever become so bright that it goes outside the range of values possible for the monitor to display.