Using shader effects

If you have played some current games you probably already know the term shaders. A simple definition is that shaders are something you can create visual effects with. But this answer is not precise enough, so let's hear the detailed one. There are two different kinds of shaders: vertex shaders and pixel shaders. Vertex shaders can be used alone, while pixel shaders are always used in combination with vertex shaders.

Usually Ultimate 3D uses the so-called fixed function pipeline to render models. The fixed function pipeline is part of Direct3D. It computes the vertex data, meaning that it calculates the vertex position in screen space, the vertex lighting, and a couple of other bits of information. Then the computed information gets passed on and is used to draw the single triangles pixel by pixel. When you are using shaders you do not use the fixed function pipeline anymore, but the programmable function pipeline. This means that the way Ultimate 3D computes the vertex data and the pixel colors can be changed in any way.

Vertex shaders are very short programs that get executed for every single vertex in every single frame to compute their data. This way you can change the geometry of objects without the mesh manipulation functions of Ultimate 3D, and you can change the way textures get mapped on the object by modifying the way the final texture coordinate gets calculated. Basically, you can realize a custom lighting engine. Pixel shaders are even shorter programs that get executed for every single pixel in every single frame to compute their data. They can retrieve the texture coordinates that have been calculated by the vertex shader and they can access textures. The result of the computations in the pixel shader is always only one single color value that will be used as color for the pixel. Note that shaders can be created and edited in any text editor, e.g. Notepad. The following picture illustrates how they work:

First the T&L or the Vertex Shader processes the vertices. Then these vertices get passed to the rasterizer. It determines which pixels need to be rendered for the triangles of the mesh. Finally the fixed function pipeline or the pixel shader calculates the pixel color and writes it to the screen. Actually, there are a few more steps in the rendering pipeline, like the fog being applied after the pixel shader, but mentioning those in the illustration would make it more complicated than necessary.

Just to give you an idea of how a shader may look, here's an example. It shows a very simple shader. It just renders the object applying its material color and its first texture. A shader effect is always made up of two or three files. A vertex shader file (*.vsh) has to be available; a pixel shader file (*.psh) is optional. There also has to be an Ultimate 3D effects file (*.ufx) that contains information about how the vertex shader and the pixel shader need to be used by Ultimate 3D.

Here's the vertex shader:

Here's the pixel shader:

And here's the Ultimate 3D effect file:

It's only seven lines of shader code and two of them are always the same. So, as you can see, shader programming is not too hard. However, it requires a little bit of mathematical knowledge. Some of this knowledge can be read up in the tutorial about the math functions of Ultimate 3D. If you skip this tutorial that's no problem. But one day you may notice that you have become a better programmer and that you feel ready for it now. And then you should come back to expand your horizon.

In modern games you see practically nothing that does not use shader effects. They are extremely important for modern graphics. Once you are able to program shaders you have the ultimate solution for all your special effect needs. Just to give you an idea of what you are missing when you skip this chapter, here are a few examples for techniques that can be realized with shaders easily:
- Reflection and refraction mapping for perfect water
- Post screen shaders for effects such as HDR and motion blurring
- Texture perturbation for great weather effects and extremely realistic fire
- Post screen warp effects
- And many more. There's probably no one who knows about all of the possible techniques.

Also, the Ultimate 3D demo contains a couple of cool shader effects. Unfortunately it would take too long to write a complete introduction into shader programming here. Fortunately other people have done this before, namely Wolfgang Engel. He has written great tutorials to get started with shader programming.

This is a great tutorial about programming vertex shaders by Wolfgang Engel.

This is another tutorial by Wolfgang Engel about pixel shader programming (note that you can not use pixel shaders without vertex shaders).

And here's a third tutorial by him that gives a nice example on shader programming by showing how to realize different per pixel lighting techniques with it.

You should not miss the Microsoft vertex shader reference and the pixel shader reference, either. It's not a tutorial but a place where you can look up anything about shader programming and where you get really reliable information.

Once you've read and understood Wolfgang Engel's tutorials, you should have the theoretical base you need to program shaders. Anyway learning by doing is the best way to learn shader programming. So just play around with it. Not every shader you are making has to fulfill some purpose as long as you learn something by writing it. If you have any questions about shader programming you can ask them at the Ultimate 3D Community for sure. To program shaders that are compatible with Ultimate 3D you need one piece of information: you need to know which vertex shader input data register contains what kind of data. So here's a little list:
v0 - Position in mesh space
v1 - Skinning weight data for models with skinning, second position for models with tweening
v2 - Four matrix indices for models with skinning, second normal for models with tweening
v3 - (First) normal vector
v7 to v14 - Up to eight one- to four-dimensional texture coordinates

Introduction to the *.ufx file format

Shader programming is difficult enough, so it would be hard if you had to care about calculating the values for the shader constants as well. Because of this, I invented the *.ufx file format. It's a file format that manages shader effects, including all information that belongs to them. If you are using a *.ufx file for your shader effect you will not have to do anything after loading it. You do not need to calculate a big amount of shader constants because you can choose from a long list of commonly used shader constants for every single constant register in the *.ufx file. And the constants that are actually constant for every usage of the shader can be set up in the *.ufx file or using the def instruction of the shader. You can also define alternative effects that are to be used if the required shader version is not supported (these are also called fallback techniques).

So you see that the *.ufx file format can save you a lot of work. Unfortunately it's not that user-friendly currently. But do not worry, I'll write an effect composer tool when I get the time. In this tool there'll be a drag and drop system to create *.ufx files and a graphics output that shows you the result immediately.

Let's get to the explanation of the *.ufx file format. Every *.ufx file must be made up of a couple of lines. Each line can contain exactly one chunk. A chunk always begins with a chunk name. Then a couple of parameters for the chunk can follow, which can be strings or numbers. Note that you can put in comments wherever you want by writing "//". The first chunk that should be in the *.ufx file is:

version
Currently there are the versions 1.0 and 1.1. Every 1.0 effect file is also a valid 1.1 shader and will get compiled accordingly. So you should always write 1.1 here.

Next you have to define the shader files that are to be used for the effect (they should be *.vsh and *.psh files). There are two different chunks for this:

FileNameWithPath
The file name and the path of the shader file relative to the folder that contains the game executable, e.g. effects\EffectVertexShader.vsh.

If you have been working with the Ultimate 3D effect file format 1.0 you may be wondering where the TextureCoordinateSize parameters have gone. The answer is that Ultimate 3D determines them automatically now, depending on the vertex data available in the model. This makes the shader system more fail-safe. If you try to apply a shader effect for a model that does not have the required vertex data an error message will pop up. After you've defined the vertex shader you can define a pixel shader:

FileNameWithPath
The file name and the path of the shader file relative to the folder that contains the game executable, e.g. effects\EffectPixelShader.psh.

Note that you do not have to set up a pixel shader. Next you can configure the constants that need to be set up for the shaders. There's only one chunk for this but it can take a long list of parameters. Understanding what the parameters mean requires an understanding of matrix and vector calculations. Here's the constant chunk for vertex shaders:

ConstantIndex
The index of the constant register this constant is to be associated with. This can be a value in the range from 0 to 95.

ConstantIdentifier
What follows is a long list of all constant registers Ultimate 3D can compute for you automatically. All matrices or vectors that refer to the direction or the position of other objects are already transformed to mesh space! It's important to keep this in mind. Transforming directions or positions into mesh space before passing them to the vertex shader is much more efficient than doing it for every single vertex in the vertex shader. And usually the position in world space is not needed. In case you should need it anyway you can get it by transforming it back using the mesh to world space transformation matrix. The exception from this are models with vertex skinning, because those have many different mesh to world space transformations. For those the constant information is passed in world space and the world to mesh space matrix and the mesh to world space matrix are always the identity matrix. Color information is always passed using values in the range from 0 to 1 instead of values in the range from 0 to 255.
VS_CONST_MANUAL_SETTABLE: This parameter tells Ultimate 3D that it's supposed to set the constant to the values given in the next parameter (ManualSettableValues).
VS_CONST_WORLD_VIEW_PROJ_TRANS_1: The first column of the product of the world, view and projection transformation which results in the mesh to projection space transformation matrix.
VS_CONST_WORLD_VIEW_PROJ_TRANS_2: The second column of the same matrix.
VS_CONST_WORLD_VIEW_PROJ_TRANS_3: The third column of the same matrix.
VS_CONST_WORLD_VIEW_PROJ_TRANS_4: The fourth column of the same matrix.
VS_CONST_WORLD_VIEW_TRANS_1: The first column of the product of the world and view transformation which results in the mesh to view space transformation matrix.
VS_CONST_WORLD_VIEW_TRANS_2: The second column of the same matrix.
VS_CONST_WORLD_VIEW_TRANS_3: The third column of the same matrix.
VS_CONST_WORLD_VIEW_TRANS_4: The fourth column of the same matrix.
VS_CONST_WORLD_TRANS_1: The first column of the mesh to world space matrix.
VS_CONST_WORLD_TRANS_2: The second column of the mesh to world space matrix.
VS_CONST_WORLD_TRANS_3: The third column of the mesh to world space matrix.
VS_CONST_WORLD_TRANS_4: The the fourth column of the mesh to world space matrix.
VS_CONST_INV_WORLD_TRANS_1: The first column of the world to mesh space matrix.
VS_CONST_INV_WORLD_TRANS_2: The second column of the world to mesh space matrix.
VS_CONST_INV_WORLD_TRANS_3: The third column of the world to mesh space matrix.
VS_CONST_INV_WORLD_TRANS_4: The fourth column of the world to mesh space matrix.
VS_CONST_VIEW_TRANS_1: The first column of the world to view space matrix.
VS_CONST_VIEW_TRANS_2: The second column of the world to view space matrix.
VS_CONST_VIEW_TRANS_3: The third column of the world to view space matrix.
VS_CONST_VIEW_TRANS_4: The fourth column of the world to view space matrix.
VS_CONST_PROJ_TRANS_1: The first column of the view to projection space matrix.
VS_CONST_PROJ_TRANS_2: The second column of the view to projection space matrix.
VS_CONST_PROJ_TRANS_3: The third column of the view to projection space matrix.
VS_CONST_PROJ_TRANS_4: The fourth column of the view to projection space matrix.
VS_CONST_MATRIX_STACK: This constant is needed for models with vertex skinning. It differs from all other constants because it does not use only one constant register, but all constant registers that are behind the defined constant register. The constant register it's set up for will contain the values x=number of passed matrices, y=constant register of first matrix, z=3, w=1. These values are needed to convert the matrix indices of v2 to constant register offsets. All constant registers behind VS_CONST_MATRIX_STACK will be overwritten with transposed 4x3 mesh to world space matrices of the single bones. You can use m4x3 to transform vectors with them.
VS_CONST_CAMERA_POS: The camera position (in mesh space).
VS_CONST_MAT_AMBIENT: The ambient color of the material.
VS_CONST_MAT_DIFFUSE: The diffuse color of the material.
VS_CONST_MAT_EMISSIVE: The emissive color of the material.
VS_CONST_LIGHT_AMBIENT_COLOR: The color of the ambient light.
VS_CONST_LIGHT0_COLOR: The light color of the first light source set up for this shader (see below).
VS_CONST_LIGHT1_COLOR: The light color of the second light source set up for this shader (see below).
VS_CONST_LIGHT2_COLOR: The light color of the third light source set up for this shader (see below).
VS_CONST_LIGHT_DIRECTIONAL_DIRECTION: The direction of the directional light source set up as first light source (see below).
VS_CONST_LIGHT_POINT0_POSITION_INV_RANGE: This parameter will make Ultimate 3D pass the mesh space position of the first light source to the x-, y- and z-components of the shader constant and the reciprocal of the light range to the w-component.
VS_CONST_LIGHT_POINT1_POSITION_INV_RANGE: Same as above but for the second light source.
VS_CONST_LIGHT_POINT2_POSITION_INV_RANGE: Same as above but for the third light source.
VS_CONST_FOG_DISTANCE: This parameter will make Ultimate 3D pass the start distance of the fog to the x-component, the end distance of the fog to the y-component, the reciprocal start distance to the z-component and the reciprocal end distance to the w-component.
VS_CONST_CURRENT_TIME: This parameter will make Ultimate 3D pass the time that has elapsed since the system was started in seconds to the x-component, the time that has elapsed since Ultimate 3D was initialized to the y-component, 0 to the z-component and 1 to the w-component.
VS_CONST_CURRENT_FRAME: This parameter will make Ultimate 3D pass the fractional part of the current frame value to the x-component and one minus the content of the x-component to the y-component. The z-component will contain 0 and the w-component will contain 1. This constant is meant to be used to implement vertex tweening.

ManuallySettableValues
This parameter is only required if you passed VS_CONST_MANUAL_SETTABLE for the previous parameter. In this case you have to give four real values here that will be set up as values for the x-, y-, z- and w-component of the shader constant.

Setting up pixel shader constants is easier, because you do not have that large a selection of different constants. Pixel shaders work on a lower level than vertex shaders and can take at most eight shader constants. That's why the choice is smaller here. Here's the chunk that's required to set up pixel shader constants:

ConstantIndex
The index of the constant register you want to set up. This value can be in the range from 0 to 7.

ConstantIdentifier
As I already said there aren't that many different identifiers as for vertex shader constants. Here's a list of them:
PS_CONST_MANUAL_SETTABLE: This parameter tells Ultimate 3D that it's supposed to set the constant to the values given in the next parameter (ManualSettableValues).
PS_CONST_LIGHT_AMBIENT_COLOR: The color of the ambient lighting.
PS_CONST_LIGHT_AMBIENT_COLOR_MATERIAL: The product of the color of the ambient light and the ambient color of the material.
PS_CONST_LIGHT0_COLOR: The color of the first light source that's set up for this effect (see below).
PS_CONST_LIGHT1_COLOR: The color of the second light source that's set up for this effect (see below).
PS_CONST_LIGHT2_COLOR: The color of the third light source that's set up for this effect (see below).
PS_CONST_LIGHT0_COLOR_MATERIAL: The color of the first light source that's set up for this shader multiplied by the diffuse color of the material.
PS_CONST_LIGHT1_COLOR_MATERIAL: The color of the second light source that's set up for this shader multiplied by the diffuse color of the material.
PS_CONST_LIGHT2_COLOR_MATERIAL: The color of the third light source that's set up for this shader multiplied by the diffuse color of the material.
PS_CONST_MAT_AMBIENT: The ambient color of the material.
PS_CONST_MAT_DIFFUSE: The diffuse color of the material.
PS_CONST_MAT_EMISSIVE: The emissive color of the material.

ManuallySettableValues
This parameter is only required if you have passed PS_CONST_MANUAL_SETTABLE for the previous parameter. In this case you have to give four real values here that will be set up as values for the x-, y-, z- and w-component of the shader constant.

These are all chunks you need to create a shader effect. But what about fallback techniques? They're actually quite easy. You just have to write:

Between the two brackets you can put in one VertexShaderFile chunk, one PixelShaderFile chunk, and as many VSConstant and PSConstant chunks as you want.

Since that was a lot of definitions here's one example for a valid *.ufx file (assuming that the shader files it refers to exist).

Applying a shader

That was a lot of theory. Now let's get back to the practical use of shader effects. Setting up shader effects is pretty easy. Every material of a model can have its own shader effect. To set it up you need only one function:

Setting up shader constants

Most required shader constants can be set up using the *.ufx files. But what about those that can not be calculated by Ultimate 3D automatically? For those, there are two simple functions to change the values of vertex and pixel shader constants that do not get set up by Ultimate 3D automatically. Here's a description of them: