Outline

• Projections, clipping, culling, viewport.
• Quiz
• Materials
• Textures

Introduction

• Last week, we said there were four stages of the graphics pipeline: Vertex Processing, Clipping and assembly, Rasterization, Fragment Processing
• We discussed vertex processing till eye coordinates and we discussed how in OpenGL to create a shader program and send it the relevant matrix.
• The kind of matrices we send would something like: `[[a,b,c,d], [e,f,g,h], [i,j,k,l], [0,0,0,1]] = [[1,0,0,d], [0,1,0,h], [0,0,1,l],[0,0,0,1]][[a,b,c,0], [e,f,g,0], [i,j,k,0], [0,0,0,1]]`
  `d,h,l` above give us a translation and `a,b,c,e,f,g,i,j,k` give us a linear transformation such as rotation or scaling. In short hand,
  `[[l,t],[0,1]]=[[i,t],[0,1]][[l,0],[0,1]]`
  or `A=TL`. If `L` is a rigid body transformation (RBT) (no reflections), we write `A=TR`.
• Some example of L are:
  `[[k_x^2v +c, k_xk_yv-k_zs, k_xk_zv + k_ys],[k_yk_xv +k_zs , k_y^2v + c , k_yk_zv - k_xs],[k_zk_xv -k_ys, k_zk_yv +k_xs, k_z^2v +c]]`,
  a rotation by `theta` about the vector `[k_x, k_y, k_z]^t` where `c = cos theta`, `s= sin theta`, and `v = 1- c` and
  `[[s_x, 0, 0], [0, s_y, 0], [0, 0, s_z]]`
  a scaling in each coordinate axis.
• Only the `L` part of the above affects the normal model view matrix:
  `[vec n, 1](tilde p - tilde p_0) = [vec n, 1](A^(-1)A)(tilde p - tilde p_0)=([vec n, 1]A^(-1))(A(tilde p - tilde p_0))`.
  The vector `([vec n, 1]A^(-1))` is normal to the points in plane after the affine transformation `A`. So the NMVM, is:
  `[[l^(-1),0],[0,1]]`
• Given our matrices in eye coordinates, let's see how we project and continue down the pipeline.

Projections

• If in the eye frame, we imagine we are looking in the negative `z_e` direction and that we are trying to project the scene onto a screen at `z_e=-1` we get the picture above.
• Using similar triangles one can show a point `tilde p =[x_e, y_e, z_e, 1]^t` maps to a point on this plane with coordinates: `x_n = - x_e/z_e` and `y_n = - y_e/z_e`.
• The above setup could be modeled with matrices as:
  `[[1,0,0,0],[0,1,0,0],[-,-,-,-],[0,0,-1,0]][[x_e],[y_e],[z_e],[1]] = [[x_c],[y_c],[-],[w_c]] = [[x_n w_n],[y_n w_n],[-],[w_n]]`
  where dash (-) means "don't care yet". In this particular, `w_n = w_c = -z_e`
• The vector `[x_c,y_c,-,w_c]^t` is called the clip coordinates of `tilde p` and `[x_n, y_n, z_n, w_n]^t` are called normalized device coordinates.
• The above is a simple example of a projection matrix.
• Normalized devices coordinates are real coordinates such that `-1 le x_n le 1`, `-1 le y_n le 1`, `-1 le z_n le 1`. I.e., they are in the so-called canonical cube. We will, via a viewport transformation, map these latter into screen coordinates.

More Projections

• We said last semester that to move the plane on which we project, we essential multiple by `n`.
• On the viewing plane, we might be interested in mapping a rectangle between `l` and `r` horizontally and `t` and `b` vertically into our canonical cube.
• We might also want to set a boundary for nearest `n` and farthest `f` things that we want in our cube.
• Modifying our matrix of the last slide to handle this gives:
  `[[x_nw_n],[y_nw_n],[z_nw_n],[w_n]] = [[x_c],[y_c],[z_c],[w_c]] = [[(2n)/(r-l),0,(r+l)/(r-l),0],[0,(-2n)/(t-b),(t+b)/(t-b),0],[0,0,(f+n)/(f-n),(2fn)/(f-n)],[0,0,-1,0]][[x_e],[y_e],[z_e],[1]]`

Quiz

Clipping and Culling

• Our vertex shader of last day applies the projection matrix of the last slide and give us their clip coordinates.
• The vertex shader might also calculate other varying variables for each vertex, such as color.
• Clipping is run on each triangle. If an edge of a triangle would leave the canonical cube (can check `-w_c < x_c < w_c`, `-w_c < y_c < w_c`, `-w_c < z_c < w_c`), its intersection with the cube is calculated. This is done by OpenGL. we gave the equation of where a line intersects a plane last semester:
  `\frac{\vec n\cdot ( \vec V_1 - \vec P_1)}{\vec n\cdot ( \vec P_2 - \vec P_1)}`.
  This process may create new vertices.
• For each vertex that we now have, and for each varying variable `v`, OpenGL create an internal variable `v/w_n`. Additionally, for each vertex, OpenGL create one internal variable `1/w_n`. This is done because these ratios linear interpolate correctly in windows coordinates.
• For each vertex, division is done to obtain the normalized device coordinates. `x_n = x_c/w_c`, `y_n = y_c/y_c`, `z_n = z_c/w_c`. (We gave an example last semester where things look wrong, say if one has edges from in front to behind the camera, if clip in normalized coordinates.)
• Backface culling might be done at this point (triangle point away from the eye are eliminated (culled) from the scene). This can be enabled via the call:
  glEnable(GL_CULL_FACE);

Triangles on the Screen, z-buffering

• For each vertex, we then map normalized coordinates to windows coordinates (this might `W` pixels wide by `H` pixels high). This is done by the viewport matrix:
  `[[x_w],[y_w],[z_w],[1]] = [[W/2, 0, 0, (W-1)/2],[0, H/2, 0, (H-1)/2],[0, 0, 1/2, 1/2],[0,0,0,1]][[x_n],[y_n],[z_n],[1]]` which is set up in OpenGL with the call:
  glViewport(0,0,W, H);
• The `[x_w, y_w]^t` coordinates are used to position the triangle on the screen.
• Rasterization is done next. OpenGL scans lines down the image. For each interior pixel of a triangle (can use half-planes test to check interior), linear interpolation is used to interpolate values `z_w, v/w_n`, and `1/w_n`.
• We write pixels to a frame buffer which includes z information. (A z-buffer). This allows us to use interpolated `z_w` values to either only keep the nearest point at that pixel or to do blending. This can be set up in OpenGL as:

  glClearColor(128./255, 200./255, 255./255, 0.);
  glClearDepth(0.);
  glEnable(GL_DEPTH_TEST);
  glDepthFunc(GL_GREATER);
  glReadBuffer(GL_BACK); //start out using back buffer
  

• At each pixel, and for all varying variables, division is done on the interpolated internal variables to obtain their correct value `v=(v/w_n)/(1/w_n)`. As this is a ratio (quotient of) linear interpolats. It is called rational linear interpolation.
• Finally, `v` passed to the fragment shader where the color of a pixel is determined.

Materials

• Last semester, we gave fragment shaders which described how materials should behave with respect to different light sources.
• A material might produce an intrinsic amount of light from all directions called ambient light.
• The material might reflect light in all directions from a point light sources. The intensity of outgoing light would then depend on the angle of the incoming light source. This is called diffuse light.
• Finally, a material might reflect some light like a mirror based on the incoming direction and the normal to the surface (we used a halfway vector). This is called specular reflection.
• We also modeled materials which were anisotropic -- where the orientation of the material about its normal also makes a difference, but let's keep things simple and look at a shader that handles, the three kinds of properties above to give an outgoing color intensity.

Fragment Shader for a Material

#version 330

uniform vec3 uLight;
in vec3 vColor;
in vec3 vNormal;
in vec4 vPosition;

out fragColor;

void main()
{
    vec3 toLight = uLight - vec3(vPosition);
    vec3 toV = -normalize(vec3(vPosition));
    toLight = normalize(toLight);
    vec3 h = normalize(toV + toLight);
    vec3 normal = normalize(vNormal);

    float specular = pow(max(0.0, dot(h, normal)), 64.0);
    float diffuse = max(0.0, dot(normal, toLight));
    vec3 intensity = vec3(0.1, 0.1, 0.1) + vColor * diffuse
        + vec3(0.6, 0.6, 0.6) *specular;

    fragColor = vec4(intensity.x, intensity.y, intensity.z, 1.0);
}

Texture Mapping

• A fragment shader can also compute a pixel's color based on data fetched from an auxiliary image called a texture.
• For a given triangle vertex in our scene, we might give a coordinate `x_t, y_t` in this image, and then use interpolation to map the portion of the image within the texture onto the triangle in the scene.
• The fragment shader can grab the texture color that is pointed to by a texture coordinate and send it to the frame buffer in different ways.

Basic Texturing Vextex Shader

#version 330

uniform mat4 uProjMatrix;
uniform mat4 uModelViewMatrix;

in vec2 aTexCoord;
in vec4 aVertex;

out vec2 vTexCoord;

void main(void)
{
    vTexCoord = aTexCoord;
    gl_Position = uProjMatrix * uModelViewMatrix * aVertex;
}

• Notice in the vertex shader we are essentially just passing along the texture coordinate. Remember the vertex shader just does things at the vertices in our vertex buffers, i.e., the vertices of our triangles.
• The interiors of our triangles will be handled in the fragment shader.

Basic Texturing Fragment Shader

#version 330

uniform sampler2d uTexUnit0;

in vec2 vTexCoord;
out fragColor;

void main(void)
{
    vec4 texColor0 = texture2D(uTexUnit0, vTexCoord);
    fragColor = texColor0;
}

• sampler2D is a special GLSL data type that refers to an OpenGL texture unit.
• The call to texture2D fetches a value from the texture unit.
• At its simplest, we just fetch r,g,b values from the texture and send them directly to the framebuffer.
• Alternatively, the texture data could be interpreted as the diffuse material color of the surface point, and we could then perform the diffuse material computation we described before.