Introduction

On Monday, we started talking about resampling.
The idea was that in applying a texture to a surface and drawing the result, we go through the following steps:
- Start with a discrete image or texture `T[k][l]`. We will use subscript `t` to denote texture coordinates, `w` to denote windows coordinates.
- Reconstruct a continuous texture `T(x_t, y_t)` using a set of basis functions `B_(k,l)(x_t, y_t)`.
- Apply the geometric warp based on triangle and camera geometry to the continuous image.
- Integrate the result against a set of filters `F_(i,j)(x_w, y_w)` to obtain the discrete output image.
If `M(x_w, y_w)` is the geometric transform in the above. Putting these steps together yields the equation:
`I[i][j] leftarrow int int_Omega dx_w dy_w F_(i,j)(x_w,y_w)sum_(k,l) B_(k,l)[M(x_w, y_w)]T[k][l]`
`=sum_(k,l)T[k][l]int int_Omega dx_w dy_w F_(i,j)(x_w,y_w)B_(k,l)[M(x_w, y_w)]`
which tells us how each pixel can be obtained as a linear combination of the input texture pixels.
On Monday, we considered tent basis functions, and I said people often use bicubic basis functions.
The above integral is happening in windows coordinates.
We start today by seeing if we can gain some insight into what happening by instead considering the integral over the texture domain.

Resampling in the Texture Domain

To reframe the last slide in the texture domain we need to do a change of variable mapping back to the texture domain.
Thankfully, the book has done this for us so we don't have to hunt for our old calculus book. Let `N = M^(-1)`, to perform the operation of the last slide with the integral in the texture domain, we calculate:
`I[i][j] leftarrow int int_(M(Omega)) dx_y dy_t |det(D_N)| F_(i,j)[N(x_t, y_t)]sum_(k,l) B_(k,l)(x_t, y_t)T[k][l]`
`= int int_(M(Omega)) dx_y dy_t |det(D_N)| F'_(i,j)(x_t, y_t)sum_(k,l)B_(k,l)(x_t, y_t)T[k][l]`
Here `D_N` is the Jacobian of `N` (matrix of partial first derivatives), and `F' = F circ N`.
When `F` is a box filter we get:
`I[i][j] leftarrow int int_(M(Omega_(i,j))) dx_y dy_t |det(D_N)| F'_(i,j)(x_t, y_t)sum_(k,l) B_(k,l)(x_t, y_t)T[k][l]`
Roughly, this says we integrate over the region `M(Omega_(i,j))` on the texture domain and blend that data together. In the above picture, we see how the texture for the image on the left is calculated from the quadrilateral on the right.
If our transformation `M` has the effect of blowing up the texture, then `M(Omega_(i,j))` has a very narrow footprint over `T(x_t, y_t)`.

When we blow up textures...

Our equation includes both a reconstruction (`sum`) and a filtering (`F`) component.
When we are blowing up a texture, the filtering component may have a minimal impact because the region `M(Omega_(i,j))` might be smaller than a unit pixel in texture space.
When this happens, there is not much detail that needs blurring/averaging and the integration step can be dropped, giving us the equation:
`I[i][j] leftarrow sum_(k,l) B_(k,l)(x_t, y_t)T[k][l]`
where `(x_t, y_t)` is `M(i,j)`
So we are just point sampling the reconstructed and transformed texture.
To tell OpenGL that we expect that the pixel being textured maps to an area less than or equal to one texture element, we use GL_TEXTURE_MAG_FILTER in our texture OpenGL call.
In this situation people typically use bilinear reconstruction, which corresponds to the GL_LINEAR texture flag in OpenGL. If we wanted to round to the nearest point (which is faster but more pixelatey) we could use GL_NEAREST.
A complete OpenGL call to set this up for a 2D (as opposed to cube map) texture would be:
```
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
```
A situation where the above would be a good choice is the pixel represented by the orange square on the last slide.

Mip Map

The opposite situation to the last slide is when we shrink down a texture.
In this case, the filtering component of our equation should not be ignored.
In fact, there may be numerous texture pixels under `M(Omega_(i,j))` and we won't be able to do a look up in our texture in constant time (as in the last slide).
If we could somehow, make the pixels in the texture "bigger", so there would be fewer of them in `M(Omega_(i,j))`, it would solve this problem.
This is the idea behind making of a mip map.
In mip mapping, we let `T^0` be our original texture. Then we make a series of lower and lower resolution (blurrier) texture `T^i`, where `T^(i+1)` is twice as blurry as `T^i`.
That is we represent `T^(i+1)` with 1/2 the number of pixels of `T^i` in both the `x` and `y` direction.
This collection called a mip map is created before any triangle rendering is done.
One (the least sophisticated) way to make `T^(i+1)` from `T^i` is to simply average successive `2 times 2` pixel blocks in `T^i` to produce a single pixel in `T^(i+1)`
Notice the number of images we create will be at most logarithmic in the size of the original texture.

Mip Maps and OpenGL and Shrinkage

Mip maps can be built by your own code and then passed to OpenGL using glTexImage2D.
Alternatively, they can be built and loaded automatically with a call to gluBuild2DMipmaps.
Since each texture in the mipmap was different resolutions, each has its own viewport transformation from canonical texture coordinates to pixel-based ones.
During texture mapping, for each texture coordinate `(x_t, y_t)` the hardware estimates how much shrinkage is going on.
This shrinkage factor is then used to select from an appropriate resolution `T^i` from the mip map.
Because we pick from a suitably low resolution texture, additional filtering is not needed, and again, we can avoid the integration step and just use:

`I[i][j] leftarrow sum_(k,l) B_(k,l)(x_t, y_t)T[k][l]`.

What happens when we switch between resolutions?

When we decide to switch from one `T^(i+1)` to `T^i` or vice-versa we might get a spatial or (if animating) temporal discontinuity.
To solve this problem we can use tri-linear interpolation.
We imagine we have at most two textures "active" at a time, say `T^i` and `T^(i+1)`. For each we use bilinear interpolation to reconstruct a color.
Then we linearly interpolate the result based on how close we are to choosing level `i` versus `i+1`.
In OpenGL, to specify that we want mip mapping with tri-linear interpolation, we can do
```
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR_MIPMAP_LINEAR);
```
Here GL_TEXTURE_MIN_FILTER says that we want to control what happens when the pixel being mapped could map to something more than a pixel in the texture, GL_LINEAR_MIPMAP_LINEAR says we are going to be using mip maps with tri-linear interpolation.

Correctness of Mip Mapping

Mip Mapping in general is not exactly a correct computation of what should be the color of the pixel.
One problem is each that each lower resolution image is created using an isotropic shrinking. I.e, we shrink the same amount in each direction.
However, during texture mapping, some regions in texture space may get shrunk in only one direction. For example, in the yellow square example a few slides back it was shrunk more horizontally than vertically.
Even with isotropic shrinking, the data in the low-resolution image only represent very specific, dyadic patterns of pixel averages from the original image. These specific averages are then blended when a texture pixel is fetched.
A better (slower) approximation can be obtained by involving more `T^i`'s in the calculation to better approximate the area of `M(Omega_(i,j))` on the texture. This approach is called anisotropic filtering and can often be enabled in an API or using a driver control panel.

Start of Ray Tracing

Ray tracing represents a different approach to rendering than the standard OpenGL pipeline.

We can think of the OpenGL rasterization-based rendering as a form of the following algorithm:

initialize z-buffer
for all triangles
   for all pixels covered by the triangle
       compute color and z
       if z is closer than what is already in the z-buffer
           update the color and z of the pixel

This algorithm is fast because each triangle in the scene is touched only once and in a predictable order.
Even Pixar uses this algorithm it is offline program Renderman.
Another property is that during the processing of a triangle, setup computation is done only once can can be amortized over all of the pixels in it.
We can soup up this basic model using fancy shading computations and by using multipass algorithms like shadow mapping.

In basic ray-tracing, we reverse the loop orders to obtain:

for all pixels on the screen
   for all objects seen in this pixel
     if this is the closest object seen at the pixel
         compute color and z
         set the color of the pixel

Advantages of Ray Tracing

We never waste computation calculating the color of an occluded object
Transparency is easier.
We can directly render smooth objects without first having to dice them into triangles.
It is also easier to render solid objects described by volumetric set operations, such as union or intersection.
It is easier to handle reflections and shadows.

Resampling

Outline

Introduction

Resampling in the Texture Domain

When we blow up textures...

Mip Map

Mip Maps and OpenGL and Shrinkage

What happens when we switch between resolutions?

Correctness of Mip Mapping

Start of Ray Tracing

Advantages of Ray Tracing