Using shader effects

If you have played some current games you probably already know the term shaders. A simple definition is that shaders are something you can create visual effects with. But this answer is not precise enough, so let's hear the detailed one. There are two different kinds of shaders: vertex shaders and pixel shaders. Vertex shaders can be used alone, while pixel shaders are always used in combination with vertex shaders.

Usually Ultimate 3D uses the so-called fixed function pipeline to render models. The fixed function pipeline is part of Direct3D. It computes the vertex data, meaning that it calculates the vertex position in screen space, the vertex lighting, and a couple of other bits of information. Then the computed information gets passed on and is used to draw the single triangles pixel by pixel. When you are using shaders you do not use the fixed function pipeline anymore, but the programmable function pipeline. This means that the way Ultimate 3D computes the vertex data and the pixel colors can be changed in any way.

Vertex shaders are very short programs that get executed for every single vertex in every single frame to compute their data. This way you can change the geometry of objects without the mesh manipulation functions of Ultimate 3D, and you can change the way textures get mapped on the object by modifying the way the final texture coordinate gets calculated. Basically, you can realize a custom lighting engine. Pixel shaders are even shorter programs that get executed for every single pixel in every single frame to compute their data. They can retrieve the texture coordinates that have been calculated by the vertex shader and they can access textures. The result of the computations in the pixel shader is always only one single color value that will be used as color for the pixel. Note that shaders can be created and edited in any text editor, e.g. Notepad. The following picture illustrates how they work:

Illustration not available

First the T&L or the Vertex Shader processes the vertices. Then these vertices get passed to the rasterizer. It determines which pixels need to be rendered for the triangles of the mesh. Finally the fixed function pipeline or the pixel shader calculates the pixel color and writes it to the screen. Actually, there are a few more steps in the rendering pipeline, like the fog being applied after the pixel shader, but mentioning those in the illustration would make it more complicated than necessary.

Just to give you an idea of how a shader may look, here's an example. It shows a very simple shader. It just renders the object applying its material color and its first texture. A shader effect is always made up of two or three files. A vertex shader file (*.vsh) has to be available; a pixel shader file (*.psh) is optional. There also has to be an Ultimate 3D effects file (*.ufx) that contains information about how the vertex shader and the pixel shader need to be used by Ultimate 3D.

Here's the vertex shader:

// This has to be the first line in every vertex shader
vs.1.1

// This code line transforms the vertex position in mesh space into
// screen space. m4x4 is the command that does this. oPos is the name
// of the output register the shader has to save the screen space
// position to. The vertex position in mesh space is always saved in
// the register v0. c0 is a variable that gets set before the model
// gets rendered, a so called constant register. Constant registers get
// defined by the *.ufx file. It contains the beginning of the
// transformation matrix that is needed to calculate the screen space
// position.
m4x4 oPos, v0, c0

// This code line simply passes the material color, by copying it from
// the fifth constant register to the diffuse color output register.
mov oD0, c4

// This code line passes the first texture coordinate of the mesh by
// copying it from the seventh vertex data register to the texture
// data output register.
mov oT0, v7

Here's the pixel shader:

// This can be ps.1.0, ps.1.1, ps.1.2, ps.1.3 or ps.1.4. If you
// use higher pixel shader versions less PCs will support the effect,
// but you will have more possibilities. This example uses ps.1.4
// because the code is simpler in this version.
ps.1.4

// Use the first texture coordinate (t0), to get a texture color from
// the first texture (r0) and save the result to the temporary register
// r0. Comment: The r0 has two purposes here: It identifies the texture
// (it would be r1 for the second texture) and it is used as output
// register.
texld r0, t0

// Multiply the texture color (r0) by the material color (v0) and save
// the result to the output register (r0). The value of r0 at the end
// of the shader code will be used as color for the pixel.
mul r0, r0, v0

And here's the Ultimate 3D effect file:

// This has to be the first line of every Ultimate 3D effect file
Ultimate3DEffectFile 1.0

// Use the vertex shader (assuming that it's saved to this file)
VertexShaderFile effects/VertexShader.vsh
// Use the pixel shader. As I already said the vertex shader could
// also be used without a pixel shader, anyway it's used here.
PixelShaderFile effects/PixelShader.psh

// c0 to c3 have to contain the correct transformation matrix in the
// vertex shader
VSConstant 0 VS_CONST_WORLD_VIEW_PROJ_TRANS_1
VSConstant 1 VS_CONST_WORLD_VIEW_PROJ_TRANS_2
VSConstant 2 VS_CONST_WORLD_VIEW_PROJ_TRANS_3
VSConstant 3 VS_CONST_WORLD_VIEW_PROJ_TRANS_4
// c4 has to contain the material color in the vertex shader
VSConstant 4 VS_CONST_MAT_DIFFUSE

It's only seven lines of shader code and two of them are always the same. So, as you can see, shader programming is not too hard. However, it requires a little bit of mathematical knowledge. Some of this knowledge can be read up in the tutorial about the math functions of Ultimate 3D. If you skip this tutorial that's no problem. But one day you may notice that you have become a better programmer and that you feel ready for it now. And then you should come back to expand your horizon.

In modern games you see practically nothing that does not use shader effects. They are extremely important for modern graphics. Once you are able to program shaders you have the ultimate solution for all your special effect needs. Just to give you an idea of what you are missing when you skip this chapter, here are a few examples for techniques that can be realized with shaders easily:
- Reflection and refraction mapping for perfect water
- Post screen shaders for effects such as HDR and motion blurring
- Texture perturbation for great weather effects and extremely realistic fire
- Post screen warp effects
- And many more. There's probably no one who knows about all of the possible techniques.

Also, the Ultimate 3D demo contains a couple of cool shader effects. Unfortunately it would take too long to write a complete introduction into shader programming here. Fortunately other people have done this before, namely Wolfgang Engel. He has written great tutorials to get started with shader programming.

This is a great tutorial about programming vertex shaders by Wolfgang Engel.

This is another tutorial by Wolfgang Engel about pixel shader programming (note that you can not use pixel shaders without vertex shaders).

And here's a third tutorial by him that gives a nice example on shader programming by showing how to realize different per pixel lighting techniques with it.

You should not miss the Microsoft vertex shader reference and the pixel shader reference, either. It's not a tutorial but a place where you can look up anything about shader programming and where you get really reliable information.

Once you've read and understood Wolfgang Engel's tutorials, you should have the theoretical base you need to program shaders. Anyway learning by doing is the best way to learn shader programming. So just play around with it. Not every shader you are making has to fulfill some purpose as long as you learn something by writing it. If you have any questions about shader programming you can ask them at the Ultimate 3D Community for sure. To program shaders that are compatible with Ultimate 3D you need one piece of information: you need to know which vertex shader input data register contains what kind of data. So here's a little list:
v0 - Position in mesh space
v1 - Skinning weight data for models with skinning, second position for models with tweening
v2 - Four matrix indices for models with skinning, second normal for models with tweening
v3 - (First) normal vector
v7 to v14 - Up to eight one- to four-dimensional texture coordinates


Introduction to the *.ufx file format

Shader programming is difficult enough, so it would be hard if you had to care about calculating the values for the shader constants as well. Because of this, I invented the *.ufx file format. It's a file format that manages shader effects, including all information that belongs to them. If you are using a *.ufx file for your shader effect you will not have to do anything after loading it. You do not need to calculate a big amount of shader constants because you can choose from a long list of commonly used shader constants for every single constant register in the *.ufx file. And the constants that are actually constant for every usage of the shader can be set up in the *.ufx file or using the def instruction of the shader. You can also define alternative effects that are to be used if the required shader version is not supported (these are also called fallback techniques).

So you see that the *.ufx file format can save you a lot of work. Unfortunately it's not that user-friendly currently. But do not worry, I'll write an effect composer tool when I get the time. In this tool there'll be a drag and drop system to create *.ufx files and a graphics output that shows you the result immediately.

Let's get to the explanation of the *.ufx file format. Every *.ufx file must be made up of a couple of lines. Each line can contain exactly one chunk. A chunk always begins with a chunk name. Then a couple of parameters for the chunk can follow, which can be strings or numbers. Note that you can put in comments wherever you want by writing "//". The first chunk that should be in the *.ufx file is:

Ultimate3DEffectFile %version

version
Currently there are the versions 1.0 and 1.1. Every 1.0 effect file is also a valid 1.1 shader and will get compiled accordingly. So you should always write 1.1 here.


Next you have to define the shader files that are to be used for the effect (they should be *.vsh and *.psh files). There are two different chunks for this:

VertexShaderFile %FileNameWithPath

FileNameWithPath
The file name and the path of the shader file relative to the folder that contains the game executable, e.g. effects\EffectVertexShader.vsh.


If you have been working with the Ultimate 3D effect file format 1.0 you may be wondering where the TextureCoordinateSize parameters have gone. The answer is that Ultimate 3D determines them automatically now, depending on the vertex data available in the model. This makes the shader system more fail-safe. If you try to apply a shader effect for a model that does not have the required vertex data an error message will pop up. After you've defined the vertex shader you can define a pixel shader:

PixelShaderFile %FileNameWithPath

FileNameWithPath
The file name and the path of the shader file relative to the folder that contains the game executable, e.g. effects\EffectPixelShader.psh.


Note that you do not have to set up a pixel shader. Next you can configure the constants that need to be set up for the shaders. There's only one chunk for this but it can take a long list of parameters. Understanding what the parameters mean requires an understanding of matrix and vector calculations. Here's the constant chunk for vertex shaders:

VSConstant %ConstantIndex %ConstantIdentifier %ManuallySettableValues

ConstantIndex
The index of the constant register this constant is to be associated with. This can be a value in the range from 0 to 95.

ConstantIdentifier
What follows is a long list of all constant registers Ultimate 3D can compute for you automatically. All matrices or vectors that refer to the direction or the position of other objects are already transformed to mesh space! It's important to keep this in mind. Transforming directions or positions into mesh space before passing them to the vertex shader is much more efficient than doing it for every single vertex in the vertex shader. And usually the position in world space is not needed. In case you should need it anyway you can get it by transforming it back using the mesh to world space transformation matrix. The exception from this are models with vertex skinning, because those have many different mesh to world space transformations. For those the constant information is passed in world space and the world to mesh space matrix and the mesh to world space matrix are always the identity matrix. Color information is always passed using values in the range from 0 to 1 instead of values in the range from 0 to 255.
VS_CONST_MANUAL_SETTABLE: This parameter tells Ultimate 3D that it's supposed to set the constant to the values given in the next parameter (ManualSettableValues).
VS_CONST_WORLD_VIEW_PROJ_TRANS_1: The first column of the product of the world, view and projection transformation which results in the mesh to projection space transformation matrix.
VS_CONST_WORLD_VIEW_PROJ_TRANS_2: The second column of the same matrix.
VS_CONST_WORLD_VIEW_PROJ_TRANS_3: The third column of the same matrix.
VS_CONST_WORLD_VIEW_PROJ_TRANS_4: The fourth column of the same matrix.
VS_CONST_WORLD_VIEW_TRANS_1: The first column of the product of the world and view transformation which results in the mesh to view space transformation matrix.
VS_CONST_WORLD_VIEW_TRANS_2: The second column of the same matrix.
VS_CONST_WORLD_VIEW_TRANS_3: The third column of the same matrix.
VS_CONST_WORLD_VIEW_TRANS_4: The fourth column of the same matrix.
VS_CONST_WORLD_TRANS_1: The first column of the mesh to world space matrix.
VS_CONST_WORLD_TRANS_2: The second column of the mesh to world space matrix.
VS_CONST_WORLD_TRANS_3: The third column of the mesh to world space matrix.
VS_CONST_WORLD_TRANS_4: The the fourth column of the mesh to world space matrix.
VS_CONST_INV_WORLD_TRANS_1: The first column of the world to mesh space matrix.
VS_CONST_INV_WORLD_TRANS_2: The second column of the world to mesh space matrix.
VS_CONST_INV_WORLD_TRANS_3: The third column of the world to mesh space matrix.
VS_CONST_INV_WORLD_TRANS_4: The fourth column of the world to mesh space matrix.
VS_CONST_VIEW_TRANS_1: The first column of the world to view space matrix.
VS_CONST_VIEW_TRANS_2: The second column of the world to view space matrix.
VS_CONST_VIEW_TRANS_3: The third column of the world to view space matrix.
VS_CONST_VIEW_TRANS_4: The fourth column of the world to view space matrix.
VS_CONST_PROJ_TRANS_1: The first column of the view to projection space matrix.
VS_CONST_PROJ_TRANS_2: The second column of the view to projection space matrix.
VS_CONST_PROJ_TRANS_3: The third column of the view to projection space matrix.
VS_CONST_PROJ_TRANS_4: The fourth column of the view to projection space matrix.
VS_CONST_MATRIX_STACK: This constant is needed for models with vertex skinning. It differs from all other constants because it does not use only one constant register, but all constant registers that are behind the defined constant register. The constant register it's set up for will contain the values x=number of passed matrices, y=constant register of first matrix, z=3, w=1. These values are needed to convert the matrix indices of v2 to constant register offsets. All constant registers behind VS_CONST_MATRIX_STACK will be overwritten with transposed 4x3 mesh to world space matrices of the single bones. You can use m4x3 to transform vectors with them.
VS_CONST_CAMERA_POS: The camera position (in mesh space).
VS_CONST_MAT_AMBIENT: The ambient color of the material.
VS_CONST_MAT_DIFFUSE: The diffuse color of the material.
VS_CONST_MAT_EMISSIVE: The emissive color of the material.
VS_CONST_LIGHT_AMBIENT_COLOR: The color of the ambient light.
VS_CONST_LIGHT0_COLOR: The light color of the first light source set up for this shader (see below).
VS_CONST_LIGHT1_COLOR: The light color of the second light source set up for this shader (see below).
VS_CONST_LIGHT2_COLOR: The light color of the third light source set up for this shader (see below).
VS_CONST_LIGHT_DIRECTIONAL_DIRECTION: The direction of the directional light source set up as first light source (see below).
VS_CONST_LIGHT_POINT0_POSITION_INV_RANGE: This parameter will make Ultimate 3D pass the mesh space position of the first light source to the x-, y- and z-components of the shader constant and the reciprocal of the light range to the w-component.
VS_CONST_LIGHT_POINT1_POSITION_INV_RANGE: Same as above but for the second light source.
VS_CONST_LIGHT_POINT2_POSITION_INV_RANGE: Same as above but for the third light source.
VS_CONST_FOG_DISTANCE: This parameter will make Ultimate 3D pass the start distance of the fog to the x-component, the end distance of the fog to the y-component, the reciprocal start distance to the z-component and the reciprocal end distance to the w-component.
VS_CONST_CURRENT_TIME: This parameter will make Ultimate 3D pass the time that has elapsed since the system was started in seconds to the x-component, the time that has elapsed since Ultimate 3D was initialized to the y-component, 0 to the z-component and 1 to the w-component.
VS_CONST_CURRENT_FRAME: This parameter will make Ultimate 3D pass the fractional part of the current frame value to the x-component and one minus the content of the x-component to the y-component. The z-component will contain 0 and the w-component will contain 1. This constant is meant to be used to implement vertex tweening.

ManuallySettableValues
This parameter is only required if you passed VS_CONST_MANUAL_SETTABLE for the previous parameter. In this case you have to give four real values here that will be set up as values for the x-, y-, z- and w-component of the shader constant.


Setting up pixel shader constants is easier, because you do not have that large a selection of different constants. Pixel shaders work on a lower level than vertex shaders and can take at most eight shader constants. That's why the choice is smaller here. Here's the chunk that's required to set up pixel shader constants:

PSConstant %ConstantIndex %ConstantIdentifier %ManuallySettableValues

ConstantIndex
The index of the constant register you want to set up. This value can be in the range from 0 to 7.

ConstantIdentifier
As I already said there aren't that many different identifiers as for vertex shader constants. Here's a list of them:
PS_CONST_MANUAL_SETTABLE: This parameter tells Ultimate 3D that it's supposed to set the constant to the values given in the next parameter (ManualSettableValues).
PS_CONST_LIGHT_AMBIENT_COLOR: The color of the ambient lighting.
PS_CONST_LIGHT_AMBIENT_COLOR_MATERIAL: The product of the color of the ambient light and the ambient color of the material.
PS_CONST_LIGHT0_COLOR: The color of the first light source that's set up for this effect (see below).
PS_CONST_LIGHT1_COLOR: The color of the second light source that's set up for this effect (see below).
PS_CONST_LIGHT2_COLOR: The color of the third light source that's set up for this effect (see below).
PS_CONST_LIGHT0_COLOR_MATERIAL: The color of the first light source that's set up for this shader multiplied by the diffuse color of the material.
PS_CONST_LIGHT1_COLOR_MATERIAL: The color of the second light source that's set up for this shader multiplied by the diffuse color of the material.
PS_CONST_LIGHT2_COLOR_MATERIAL: The color of the third light source that's set up for this shader multiplied by the diffuse color of the material.
PS_CONST_MAT_AMBIENT: The ambient color of the material.
PS_CONST_MAT_DIFFUSE: The diffuse color of the material.
PS_CONST_MAT_EMISSIVE: The emissive color of the material.


ManuallySettableValues
This parameter is only required if you have passed PS_CONST_MANUAL_SETTABLE for the previous parameter. In this case you have to give four real values here that will be set up as values for the x-, y-, z- and w-component of the shader constant.


These are all chunks you need to create a shader effect. But what about fallback techniques? They're actually quite easy. You just have to write:

FallBackTechnique{
}

Between the two brackets you can put in one VertexShaderFile chunk, one PixelShaderFile chunk, and as many VSConstant and PSConstant chunks as you want.

Since that was a lot of definitions here's one example for a valid *.ufx file (assuming that the shader files it refers to exist).

// Here's an example *.ufx file
Ultimate3DEffectFile 1.0

// The pixel shader below does use pixel shader model 1.4
VertexShaderFile shaders/ShaderEffect1_4.vsh
PixelShaderFile shaders/ShaderEffect1_4.psh

VSConstant 0 VS_CONST_WORLD_VIEW_PROJ_1
VSConstant 1 VS_CONST_WORLD_VIEW_PROJ_2
VSConstant 2 VS_CONST_WORLD_VIEW_PROJ_3
VSConstant 3 VS_CONST_WORLD_VIEW_PROJ_4

VSConstant 4 VS_CONST_MAT_DIFFUSE

// The shader needs these values to calculate something. They could be also set up with the def instruction of the vertex shader.
VSConstant 5 VS_CONST_MANUAL_SETTABLE 3.14 6.28 -1 5

// This fallback technique will be used if pixel shader model 1.4 is not supported
FallBackTechnique{
// The pixel shader below uses pixel shader model 1.2
VertexShaderFile shaders/ShaderEffect1_2.vsh
PixelShaderFile shaders/ShaderEffect1_2.psh
VSConstant 0 VS_CONST_WORLD_VIEW_PROJ_1
VSConstant 1 VS_CONST_WORLD_VIEW_PROJ_2
VSConstant 2 VS_CONST_WORLD_VIEW_PROJ_3
VSConstant 3 VS_CONST_WORLD_VIEW_PROJ_4

VSConstant 4 VS_CONST_MAT_DIFFUSE

VSConstant 5 VS_CONST_MANUAL_SETTABLE 3.14 6.28 -1 5
}

Applying a shader

That was a lot of theory. Now let's get back to the practical use of shader effects. Setting up shader effects is pretty easy. Every material of a model can have its own shader effect. To set it up you need only one function:

This function loads the given *.ufx file and uses it for the given material of the object that calls the function.

LoadMaterialEffect(
MaterialIndex,
UFXFile
)

MaterialIndex
The index of the material you want to apply the shader effect for.

UFXFile
The *.ufx file that is to be used for this material. If you pass an empty string for this parameter ("") the current effect will be disabled.


If you have been using vertex shaders with Ultimate 3D v. 1.31 you may remember that there was no *.ufx file format. Instead you just loaded the shader and it got a set of preset shader constants. Since I did not want to break the downwards compatibility completely, there is a function in Ultimate 3D 2.0 and higher, which works in a similar fashion. It does not ask for a *.ufx file, but for a *.vsh file.

This function sets up the given vertex shader for the given material using the following set of constants:
0: VS_CONST_VIEW_TRANS_1
1: VS_CONST_VIEW_TRANS_2
2: VS_CONST_VIEW_TRANS_3
3: VS_CONST_VIEW_TRANS_4
4: VS_CONST_WORLD_VIEW_PROJ_TRANS_1
5: VS_CONST_WORLD_VIEW_PROJ_TRANS_2
6: VS_CONST_WORLD_VIEW_PROJ_TRANS_3
7: VS_CONST_WORLD_VIEW_PROJ_TRANS_4
8: VS_CONST_WORLD_TRANS_1
9: VS_CONST_WORLD_TRANS_2
10: VS_CONST_WORLD_TRANS_3
11: VS_CONST_WORLD_TRANS_4
12: VS_CONST_MAT_DIFFUSE

CreateMaterialVSEffect(
MaterialIndex,
VertexShaderFile
)

MaterialIndex
The index of the material the vertex shader effect is to be applied to.

VertexShaderFile
The file name and path of the vertex shader that is to be used (e.g. "effects\VSEffect.vsh").


I do not recommend using this function unless you want to use shaders that are made for Ultimate 3D v 1.31. If you aren't, you should create a *.ufx file. The great thing about *.ufx files is that they actually do most of the work for you. If you want to support the Ultimate 3D Community and know how to write shader effects you should make a package of nice shader effects including the *.ufx files and post it on the Ultimate 3D Community. Having a nice collection of premade shader effects there would be great.


Setting up shader constants

Most required shader constants can be set up using the *.ufx files. But what about those that can not be calculated by Ultimate 3D automatically? For those, there are two simple functions to change the values of vertex and pixel shader constants that do not get set up by Ultimate 3D automatically. Here's a description of them:

This function sets up a new value for the vertex shader constant register with the given index, in case it does not get set up by Ultimate 3D automatically.

SetMaterialEffectVSConstant(
MaterialIndex,
VSConstantIndex,
XValue, YValue, ZValue, WValue
)

MaterialIndex
The index of the material that uses the effect.

VSConstantIndex
The index of the constant register you want to modify. This value has to be in the range from 0 to 95.

XValue, YValue, ZValue, WValue
The new values for the constant register. They can be arbitrary real values.


The function SetMaterialEffectPSConstant(...) does exactly the same thing for pixel shader constants. The only difference is that the constant index has to be in the range from 0 to 7. For that reason I'm not going to explain it explicitly here.

Earlier in the chapter, I said 'see below' a few times when talking about constant registers that give information about light sources. So see here. You can set up three light sources that are to be used for the shader effect. To do this there's the following function:

This function sets up the three light sources that are to be used for the shader effect of the given material of the model object.

SetMaterialEffectLightSource(
MaterialIndex,
LightSourceID1, LightSourceID2, LightSourceID3
)

MaterialIndex
The index of the material the light sources are to be set up for.

LightSourceID1, LightSourceID2, LightSourceID3
The Game Maker ID of the light sources that should be used for the shader effect. You can enter the id of the objects for these parameters. If you do not need all three light sources just put 0 for the ones you do not want.


If you have read the whole tutorial and the tutorials about shader programming you should be really happy. You now know how to use the most flexible and widely usable feature of Ultimate 3D. Do not underestimate this. At first it may be that you do not know what to do with shaders, but soon you'll get some ideas. Besides, there are tons of tutorials on the Internet about really great effects that can be realized using shaders. Usually there's even an example given. I recommend playing around with shaders a lot to learn them. It's really the best way to collect experience here. Not every effect you make must make sense, as long as you learn something from making it. Just by changing single lines in the shader code you can get interesting effects you couldn't make without shaders. For example, you can enter the vertex normal as diffuse color for the vertex (in the vertex shader).

So have fun :D .



© Christoph Peters. Some rights reserved.

Creative Commons License XHTML 1.0 Transitional