Graphics Refresher

CS116b/CS216

Chris Pollett

Jan 27, 2014

Outline

Graphics Pipeline
Linear and Affine Transformations
Next day, we will review projections, shaders and OpenGL.

Introduction

Today, we are trying to see/remember some of the main things we covered in CS116a.
In computer graphic, a typical scene we are trying to draw is made up of a collection of objects.
We are usually interested in drawing the surface of an object.
We assume our surfaces are smooth enough so that a fine scale they are relatively flat and can be split into triangles.
So we split out objects into triangles, record the vertices of the triangles, as well as color texture information needed to draw within a triangle.
This is the geometry of a scene.
A graphics pipeline is used to go from the geometry consisting of raw vertices and auxiliary data structures to an image one might see on the screen.

Graphics Pipeline

The computer graphics pipeline can be split into four main parts:
1. Vertex Processing - in this stage we mainly want to compute a position and a color for each vertex in our geometry.
2. Clipping and assembly - in assembly, we group our raw vertices into triangles, etc and for each triangle determine what portion is in the viewing volume. i.e., will project to something on the screen.
3. Rasterization - this computes which pixels in the frame buffer will correspond to points inside a triangle we want to draw. It generates fragments which can be thought of as a potential pixel that carries with it color and depth information that can be used to update a value in the frame buffer.
4. Fragment Processing - this takes the fragments generated by the rasterizer and actually updates values in the frame buffer. Besides overwriting more distant fragments values with nearer ones, it also might involve color blending.

Vertex Processing

To do vertex processing, we need to be able to specify points/vertices and the relationships between them.
We said our points and vertices came from a space called affine space.
We used the notation `tilde{p}` to represent a point. We called the motion that would take one point to another point, a vector `vec{v}`. i.e., `tilde{p} - tilde{q} = vec{v}`.
We imagine our vectors as living in 3D, and we can write them with respect to some basis `b_1`, `b_2`, `b_3` for 3-space. (For example, the x,y,z coordinate directions).
Each point `tilde{p}` from a collection of points `G` of a geometry can be thus expressed in terms of some origin point `tilde{o}` for that geometry as:
`tilde{p} = tilde{o} + sum_ic_ivec b_i = [vec b_1, vec b_2, vec b_3, tilde o][[c_1],[c_2],[c_3],[1]] = vec bb f^t bb c`
`[vec b_1, vec b_2, vec b_3, tilde o]` is called an affine frame, `vec bb{f}^t`, and `[c_1, c_2, c_3, 1]^t` are coordinates of the point with respect to this frame..
An affine transformation is a map from one such frame to another.
Examples of such transformations are rotations, translations, and scalings. Each of these can be expressed as 4x4 matrices which can be multiplied against our coordinates in one frame to give coordinates in the new frame.

More on Vertex Processing

We might use a world frame to describe a whole scene.
A particular object like a robot might have its vertices specified in its own frame called the object frame.
To draw an object into the world we use an affine transformation to map the vertices in the object frame into the world frame.
To draw the same robot but at a different location, we could use a second affine transformation.
After getting each of the vertices of our objects into world coordinates, we would typically use another transformation to view the scene from a particular frame (the eye frame).
The matrix that handles all of these transformations is called the model view matrix.

Matrix Stack

To build a complicated objects like a robot, it is often easier to decompose it into parts: the torso, the head, the arms, the legs.
These parts may further be decomposed into smaller parts: The arm is made of the upper arm, the lower arm, the hand, and so on.
Affine transformations facilitate in being easily able to carry out such decompositions.
Each subobject may haves its own orthonormal frame (and scaled frame) `vec bb a^t`.
We then store the vertex coordinates of the subobjects in its own coordinate system.
For example, a robot with moving limbs may use an object and scaled object frame for its body, may have a sub-object frame to represent each rotatable shoulder, and have a sub-sub-object frame to represent an upper arm for each shoulder, have a sub-sub-sub-object for an elbow joint, and, finally, a sub-sub-sub-sub-object for the lower arm. Each of these may be scaled.

The same robot after a motion in the object frame

Moving Hierarchies Example

Consider our robot. `vec bb a^t` is the right shoulder's frame, `vec bb b^t = vec bb a^t B` is the right upper arm's frame, `vec bb b'^t = vec bb b^t B'` right upper arm's scaled frame, `vec bb c^t = vec bb b^t C` is the right elbow frame, `vec bb d^t = vec bb c^t D` the lower arm frame, and `vec bb d'^t = vec bb d^t D'` the scaled right lower arm frames.
Notice each frame is obtained from the previous by adding one matrix to the list of matrices so far (ignoring scaling frames).
To move the entire object, we update its `O` matrix.
To bend the right arm at the shoulder, we update the `A` matrix. To bend the right arm at the elbow we update the `C` matrix.
To keep track of which object we are manipulating we use a matrix stack.
push(M) adds a new matrix to the top of the stack. This correspond to descending down to subobject.
pop() removes the top matrix from the stack, this corresponds to going from a subobject's frame to its parent object's frame.

Moving Hierarchies - Pseudocode

Using the stack idea we could draw the robot using the following pseudo-code:

...
matrixStack.initialize(inv(E));
matrixStack.push(O);
    matrixStack.push(O');
        //draw body
        draw(matrixStack.top(), cube); 
    // get rid of scale matrix O'
    matrixStack.pop(); 

    matrixStack.push(A);
        matrixStack.push(B);
        matrixStack.push(B');
            //draw upper arm
            draw(matrixStack.top(), sphere);
        // get rid of scale matrix B' 
        matrixStack.pop(); 

        matrixStack.push(C);
            matrixStack.push(D);
                matrixStack.push(D');
                    //draw lower arm
                    draw(matrixStack.top(), sphere); 
                // get rid of scale matrix D'
                matrixStack.pop(); 
            
            matrixStack.pop(); // bye-bye D
        matrixStack.pop(); //bye-bye C
    matrixStack.pop(); //bye-bye B
matrixStack.pop();//bye-bye A
...//draw left arm, etc.

When we push a matrix `M'` onto the top of the stack we actually peek at the matrix `M` that is currently at the top of the stack, compute `M cdot M'` and this is the matrix that actually get put on top of the stack. After the operation the top two elements of the stack would be `M` and `M cdot M'`. If we pop off of the stack at this point `M` would again be the top of the stack.