Melting Paint

Grass Demo

Uses for MutliPaint might include complex armor built of inlaid metals, woods, and stones, all modeled on a single simple poly mesh; buildings composed of multiple types of stone, glass, and metal, expressed as simple cubes; cloth with inlaid metallic threads; or as in this demo, metal partially-covered with peeling paint.

Using multiple BRDFs is common in the offline world, but rarely optimized -- instead two different shaders may be evaluated and their results blended using a mask texture or chained through "if" statements. For maximum realtime performance, MultiPaint instead integrates all of the key parts of the BRDFs as multiple painted textures, so that only one pass through the shader is required to create the mixed appearance. This permits a single-pass shader containing diffuse, specular, and environmental lighting effects in a compact, fast-executing package.

The demo permits varying the index of refraction, the depth and density of the lens. Note that the choice of geometry is arbitrary -- this sample is a sphere, but any polygonal model can be used.

This shader performs matrix palette skinning with vertex diffuse lighting. Each vertex has four indices which reference the four bones (input as uniform parameters) which influence this particular vertex, along with the weights of influence. We calculate the vertex position in world space as influenced by each bone (so, four separate positions are generated per vertex) and then we blend these four positions according to the weights to generate the final vertex position in world space. Since, in this particular mesh, the first two bones influencing each vertex are by far the most significant (have the largest weights), we only transform the normals by the first two bones and their respective weights, and then normalize the result. This normal is then used with the uniform parameter for the light vector in world space to calculate diffuse intensity.

This example renders a fogged landscape with two types of fog.  It shows the interaction of fog calculations with pixel and vertex shader programs.

The first fog factor is a traditional screen depth based fog, calculated from the Z component of the vertex after the model-view-project transform.  This value is written to the shader oFog.x which is used in a blending stage after the pixel shader program.

The second fog term is based on the model space landscape height and appears as the white fog in the valleys.  You can change the scaling of this second term with the “<” and “>” keys.  The value is passed into the pixel shader program, where it is added to the base texture color.

The landscape height is generated as the sum of three blurred and scaled random noise fields.  The blurring produces features of varying roughness, and the more blurred components are added in with a greater scale factor.

This example shows basic diffuse and specular lighting calculations, based on the Phong illumination model.  The diffuse term is calculated using the usual N dot L formulation, and the specular term uses the Blinn formulation of N dot H.  This is also the example that is used in "A Brief Tutorial", which can be found in the Cg User's Manual.

This effect computes diffuse and specular lighting in two passes using a normal from a normal map. The first pass lays down the diffuse component, by transforming the light vector into tangent space in the vertex program and then using this light vector along with the normal and a decal texture to compute diffuse lighting. In a second pass, the half-angle vector is computed per-vertex, transformed into tangent space, and passed down in a texture coordinate. H dot N is then computed per-pixel, and the result is squared a number of times to achieve a higher specular power, then modulated by a self-shadowing term and added into the framebuffer.

This demonstrates diffuse and specular illumination on a bump mapped model.  N · L and N · H are computed for each rendered pixel.  The values of the dot-products are then used as (u,v) coordinates to address an irradiance map.  This is a texture which holds a diffuse color ramp in one axis, and a specular color ramp in the alpha channel of the other axis.  In this way, an arbitrary response to the incident light direction is possible.  You can achieve any specular exponent or arbitrary function by editing the color ramps in a paint program.

The demo includes right-button menu options for loading normal map bump maps, base textures, and irradiance maps, as well as the ability to visualize the incident light vector or each texture map on its own.

This effect does per-pixel bump mapping and makes every bump cast a shadow using the horizon mapping technique.  Based on the height map, 8 horizon maps are precomputed corresponding to 8 main light directions in the tangent plane.  Every horizon map texel holds the cosine of the angle between the normal and the light direction below which this texel is shadowed by its neighboring texels.  At run-time, we assume that the surface is locally planar enough that the visibility pre-computation holds true.  For every pixel, we compare the actual cosine of the angle between the normal and the current light direction to the limit stored in the horizon map and determine if the pixel is in shadow or not.  The current light direction falls in-between two of the 8 main light directions; the value of the limit is thus computed by linear interpolation between those 2 main directions.

This shader demonstrates how to perform the ubiquitous "bumpy shiny" effect in Cg, where a per-pixel normal is used to compute a reflection vector to lookup into a cubemap. The vertex program performs the bumpy-shiny setup, by passing a matrix to transform a vector from tangent space to "cube map space" (usually world space) as texture coordinates, along with an eye vector in world space. The pixel shader then computes the texCUBE_reflect_eye_dp3x3 function, which transforms the normal fetched from a normal map by the 3x3 interpolated matrix, computes a reflected vector using the eye vector and transformed normal, and uses this reflected vector to lookup into a cubemap.

This demo shows a technique for combining normal maps at runtime. There is a "detail" normal map, which shows fine detail and fades in as you get closer to the surface, and a larger scale primary normal map. The vertex program does standard setup for tangent-space bump-mapping, and passes a light vector in tangent space to the pixel shader. At the pixel level, we combine the two normals together, which results in a non-unit length normal, and then renormalize the normal using a fast single iteration Newton-Raphson approximation. This normal is then used to calculate the diffuse intensity, which is then modulated by the base texture.

The demo is meant to show two main things:

1) Coupling two dynamic rendered-texture water simulations together so that a tiled texture applied across a large area can transition seamlessly to a local unique detail texture.

2) A method of rendering water reflections similar to Environment Mapped Bump Mapping (EMBM) but using vertex shaders and the pixel shader texm3x2tex operation for added control and realism.

The coupling of two water simulations allows a large area of procedural water to be created while still allowing for unique local features.  The reflection method used performs the same basic calculation as DX6-style EMBM, but in this case the 2x2 rotation matrix and base texture coordinates are calculated in a vertex shader and can vary per-vertex.  The water simulation is performed entirely on the graphics hardware using pixel shaders. Nearest-neighbor differencing and sampling at each texel is accomplished using a combination of vertex and pixel shaders.  

This demo shows the construction and animation of leaves of grass being calculated on the GPU. Each leaf is created by a Bezier curve with a few control points that are procedurally moved around to accommodate the wind. The normals of the leaves is also calculated from this Bezier curve. The vertices being sent to the shader consist of one degenerate quad strip for each leaf, with all the vertices collapsed to the root point of the leaf, but each vertex has a different t parameter which determines where on the Bezier curve the vertex should be located.

A flat grid of polygons is deformed by a sine wave, sampled from the constant memory using a Taylor series.  The shader calculates the first 4 values in the series.  The distance from the center of the grid is used to generate a height, which is applied to the vertex coordinate.  A normal is also generated and used to generate a texture coordinate reflection vector from the eye, which looks into a cubic environment map.

The input data for this shader is simply 2 floating-point values.  The shader generates all normal, texture and color information itself.  The menu-option allows rendering in wire frame, and left clicking allows rotating the mesh.

This demo gives the appearance that the viewer is surrounded by a large grid of vertices (because of the free rotation, but by switching to wireframe, or by increasing the frustum angle, it becomes apparent that the vertices are a static mesh, with the height, normal and texture coordinates being calculated on he fly based of the direction and height of the viewer. This technique allows for very GPU friendly water animations since the static mesh can be precomputed.

This demo illustrates how texture shaders can be used to create sprites with real depth data.  The depth sprites differ from normal sprites in that they can realistically intersect in 3D with other depth sprites or standard 3D objects.  The register combiners are used to normalize the light vector, calculate its reflection off the normal mapped surface, as well as calculate both diffuse and specular lighting equations.

This demo shows how to implement each of the standard OpenGL lighting types in the vertex shader.  It includes examples of spotlights, local lights, and infinite lights.                                                    

This shader performs diffuse and specular lighting in two passes using an interpolated normal, and shows a couple of different ways to perform per-fragment vector normalization. In the first pass, diffuse lighting is calculated as follows: in the vertex program, a light vector and normal are calculated and passed to the pixel program. At the pixel level, the light vector is normalized using a Newton-Raphson approximation and the light vector is normalized using a normalization cubemap. The normal, light vector, and decal texture are then used to compute the diffuse lighting term. In a second pass, the vertex program calculates the half-angle vector and normal, and the pixel program normalizes these two vectors, performs H dot N, and squares the result a number of times to achieve a higher specular power. This value is then added into the framebuffer.

To perform the convolution of glow source pixels into bright blurry glow, several rendering passes may be used. On GeForce3 and GeForce4, each pass accumulates four samples of the convolution kernel. Each sample is multiplied by a coefficient to determine the shape of the glow, and this shape can be any arbitrary function. To save passes, a separable convolution is used which blurs first horizontally, and then blurs the horizontal blur vertically. This may or may not be close to a separable Gaussian convolution. If the desired blur size is NxN texels, this approach reduces the number of samples which must be accumulated from N*N to 2*N, a substantial savings. This approach sacrifices some flexibility in determining the shape of the blur.

The effect runs from about 200 to 650 fps in windowed mode. Most of the time for rendering in the windowed mode is spent in blitting from backbuffer to the windowed area. To make this less significant, reduce the size of the window. This blit() time is not required for fullscreen mode, so the effect will run faster in a fullscreen application.

This example uses the vertex program Perlin noise implementation to generate a two octave ridged multifractal terrain. The geometry sent to the hardware is a static flat quadrangle mesh, which is displaced on the fly by the programmable vertex hardware. A 3D texture is used to color the terrain based on height. The sliders can be used to modify the frequencies and amplitudes of the 2 noise octaves, producing different styles of terrain.

Reference: "Texturing and Modeling, A Procedural Approach", Ebert et. al

This example demonstrates an implementation of Perlin noise using vertex programs. An animated 3D noise function is used to displace the vertices of a sphere along the vertex normal. The geometry is entirely static, but is displaced on the fly by the vertex program hardware. Perlin noise is implemented for the vertex program profile using recursive lookups into a permutation table stored in constant memory. The size of this table determines the period at which the noise function repeats. 3D noise costs around 65 instructions, 2D noise around 45, 1D noise around 20.

Reference: http://mrl.nyu.edu/~perlin/doc/oscar.html

When light strikes a boundary between different media, for example, air and glass, some of the light gets refracted and some gets reflected.  The amount of reflection depends on the ratio of refraction-indices of the two media, the polarization of the light, the wavelength of the light, and the angle of incidence of the light.  Fresnel’s formula provides an accurate description of how much light reflects at the boundary:

(1)        R(q) = ⅛ (sin²(q - q_t) / sin²(q + q_t)) (1 + cos²(q - q_t) / cos²(q + q_t)) 

We are using two approaches to approximate the Fresnel reflection formulae: per-pixel and per-vertex.  Both compute a reflection vector per-vertex and use it to look up a reflection value per-pixel via a cubic environment-map.  The Fresnel reflection value then blends this reflection-value with a material-color.  The per-pixel approximation derives the Fresnel reflection value from a texture look-up.  It computes cos(q) per-vertex -- cos(q) = N·E is readily available in the vertex-shader, as it is required in computing the reflection-vector -- and uses the interpolated per-pixel cos(q) in a 1D texture look-up that encodes R(arcos(x)). The per-vertex approximation of Fresnel reflection computes the Fresnel term R(q) per vertex.  This per-vertex term R(q) is then linearly interpolated and applied per-pixel.  Encoding equation (1) in a vertex-shader is straightforward, but also sub-optimal in terms of number of vertex-shader instructions and thus performance.  Approximating R(q) as

(2)       R(q) » R_a(q) = R(0) + (1-R(0)) (1-cos(q))⁵
instead yields good results.
See the white-paper “Fresnel Reflection” on http://www.nvidia.com/developer for details.

The menu options allow switching between the per-pixel and per-vertex approximation, as well as switching to wire-frame rendering.  Different indices of refraction are accessible via the menu or the +/- keys.

This demo calculates a reflection and a refraction vector based on the normal on the surface of the object and modulates them based on Fresnel properties.  See the white-paper “Fresnel Reflection” on http://www.nvidia.com/developer for more information about Fresnel effects.  

The code calculates three different refraction vectors for the red, green and blue wavelengths of light, each with slightly different indices of refraction. Each of these vectors is used to index into a cube map of the environment, and the resulting colors are modulated by red, green and blue and then summed to produce the rainbow effect. A reflection vector is also calculated, and used to index into the same cube map, making a total of four texture lookups. The reflection is modulated by a Fresnel approximation, which makes surfaces facing perpendicular to the viewer appear more reflective.

Reference: http://www.botzilla.com/house/RayPS.html

This example shows the usage of hardware shadow maps. A render-to-texture to a shadow map is done from the light's point-of-view, updating only depth (no color). This shadow map is then set as a texture when rendering the scene normally. The vertex shader computes the necessary texture coordinates to sample the correct texel from the shadow map given the current position, along with the current depth at that point to compare with the value in the shadow map. At the pixel level, a simple fetch from the shadow map will automatically perform a comparison between the value in the shadow map and the (r / q) texture coordinate, returning black when in shadow and white when in the light. This shadow term is then used to modulate the lighting.

Learn more about shadow mapping at http://developer.nvidia.com/view.asp?IO=shadow_mapping.

This demo shows a brute force approach to creating soft shadows in graphics hardware. It applies several stencil shadow volume passes to the scene from different light positions to approximate an area light. Each shadow volume pass creates a faint hard edged shadow, and the accumulation of many faint shadows blends into an accurate soft shadow. The technique is well suited to creating realistic soft shadows from area light sources, though the required number of passes may be prohibitive for real-time interactivity. Still, using hardware acceleration of the shadow volume creation and rendering greatly improves the speed at which frames may be rendered. Shadow volumes are created on the GPU by the same technique demonstrated in our "Stencil Shadow Volumes" demo.

To learn more about shadow volumes, check out http://developer.nvidia.com/view.asp?IO=robust_shadow_volumes.

In order for the vertex shader to extrude objects with sharp features to the proper shadow volume shape, additional triangles and vertices must be added along sharp edges. The demo code has a simple class to add such triangles and vertices to geometry where needed. This extra geometric data is required only when rendering the extruded shadow volumes. It can be skipped when rendering the objects normally to the screen.

To learn more about shadow volumes, check out http://developer.nvidia.com/view.asp?IO=robust_shadow_volumes.

This effect shows how arbitrary ad-hoc lighting models can be encoded in textures. Here, we have a texture with a bright diagonal area that encodes and anisotropic lighting-type effect. The vertex program calculates H dot N for the texture coordinate for one axis of a texture and L dot N for the other axis. End result is a bright stripe in areas where H dot N and L dot N are roughly equal.

This example shows a volume rendering technique similar to the shell rendering technique from "Real-Time Fur Over Arbitrary Surfaces" by Lengyel et al, except by rendering four layers in a single pass, with eight total layers in only two passes. The first pass computes the texture coordinates for the first four steps along the eye vector in a vertex program. At the pixel level, we use these texture coordinates to look up into four textures representing the first four layers of grass, blend them together based on the density at each pixel, and output this intermediate color to the framebuffer. In a second pass, the final four steps along the eye ray are computed, the last four layers of grass texture are blended together, and the alpha blender is setup to correctly combine these two passes to result in the final rendering of eight layers of grass. An additional tweak is provided to cap the z-component of the eye vector in the vertex program, since this has a tendency to grow extremely large and cause aliasing as the surface is viewed edge-on.