Outline

• Constant and Bilinear Reconstruction
• Quiz

Introduction

• Last week we started talking about the topic of reconstruction.
• Given a discrete image `I[i][j]`, how do we reconstruct a continuous image `I(x,y)`.
• This problem is important for the problem of resizing texture images.
• For example, in our fragment shader, we might want to fetch a color from a texture coordinate that falls in between the texture's pixels.
• Figuring out what texture color to use is called reconstruction.

Constant Reconstruction

• Suppose we have an `(x,y)` location where `x` and `y` are fractional values.
• One approach, called constant reconstruction, to calculate `I(x,y)` is to pick the closest discrete pixel and use its value.
• In pseudocode, this would be:

  color constantReconstruction(float x, float y, color image[][]) {
      int i = (int)(x + 0.5);
      int j = (int)(y + 0.5);
      return image[i][j];
  }
  

• Here "(int)" typecast rounds a number `p` to the nearest integer no larger than `p`.
• To get a feel for the behavior of this approach, note that a reconstructed image will have constant value of `I[0][0]` in the square-shaped region: `-0.5 < x < 0.5` and `-0.5 < y < 0.5`. In general, each pixel has a `1 times 1` influence region.

Bilinear Reconstruction

• The left image above is an example of using constant reconstruction to blow up a small image of a squirrel into a large one (say for texture map purposes). Os one can see it is quite blocky.
• Bilinear Reconstruction is obtained by applying linear interpolation in both the horizontal and vertical directions.
• The right image above makes use of bilinear reconstruction and looks somewhat better than the left image.

Pseudo-code for Bilinear Reconstruction

color bilinearReconstruction(float x, float y, color image[][]) {
    int intx = (int)x;
    int inty = (int)y;
    float fracx = x - intx;
    float fracy = y - inty;

    color colorx1 = (1 - fracx) * image[intx][inty] +
        (fracx) * image[intx + 1][inty];
    color colorx2 = (1 - fracx) * image[intx][inty + 1] +
        (fracx) * image[intx + 1][inty + 1];

    color colorxy = (1 - fracy) * colorx1 +
        (fracy) * colorx2;

    return (colorxy);
}

• The idea is we consider the `2 times 2` pixel square with lower left corner `(lfloor x rfloor, lfloor y rfloor)`.
• fracx says how much `(x,y)` is away from this coordinate in the `x` direction; fracy does the same for the `y` direction.
• We first linear interpolate using fracx the lower pair of pixels then the upper pair of pixel, then we take these two interpolation and linear interpolate them using fracy.
• Notice that at integer coordinates `I(i,j) = I[i][j]`.

Quiz

A Closer Look at Bilinear Reconstruction

• Consider coordinates `i < x < i+1` and `j < y < j +1` for some fixed `i` and `j`.
• Over this square, reconstruction then becomes the map: `I(i + x_f, j + y_f) leftarrow (1 - y_f){(1 - x_f)I[i][j] + (x_f)I[i+1][j]}`
  ` + (y_f){(1 - x_f)I[i][j + 1] + (x_f)I[i+1][j + 1]}`
• Here `x_f`, and `y_f` are fracx and fracy from the pseudo-code. Rearranging yields
  `I(i + x_f, j + y_f) leftarrow I[i][j]`
  `+(-I[i][j] + I[i+1][j])x_f`
  `+(-I[i][j] + I[i][j + 1])y_f`
  `+(I[i][j] - I[i][j + 1] - I[i + 1][j] + I[i + 1][j + 1])cdot x_f cdot y_f`
• That is, the reconstructed function has terms that are constant, linear, and bilinear terms in the variables `x_f, y_f` and hence also in `(x,y)`.
• This is where the name bilinear comes from. Also, the above shows we could have interpolated first vertically then horizontally, rather than the way we did and still got the same result as the function is symmetric.

Basis Functions

• We can rewrite our first equation of the previous slide to see that for a fixed position `(x,y)`, the color of the bilinear reconstruction is linear in the discrete pixel values of `I`.
  `I(i + x_f, j + y_f) leftarrow (1 \ - x_f \ - y_f \ + x_f \ cdot y_f)I[i][j]`
  `+ (x_f \ - x_f cdot y_f)I[i+1][j]`
  `+ (y_f \ - x_f cdot y_f)I[i][j+1]`
  `+ (x_f cdot y_f)I[i+1][j+1]`
  
• So we have
  `I(x, y) leftarrow sum_(i,j)B_(i,j)(x,y)I[i][j]`
  for some appropriate choice of functions `B_(i,j)(x,y)` which we call basis functions.
• Basis functions describe how much pixel `i,j` influences the continuous image at `[x,y]^t`.

Tent Functions

• In the case of bilinear reconstruction the basis functions are called tent functions.
• To define them carefully, we first define the hat function `H_i(x)` as
  `H_i(x) = {(x - i +1,text( for ) x in (i-1, i) ),(-x + i +1,text( for ) x in (i, i +1 ) ),(0,text(otherwise)) :}`
  
• The image above show how we can linearly interpolate adjacent pairs of hat functions to continuously interpolate sets of values defined on the integers.
• The tent function is defined from the hat function as:
  `T_(i,j)(x,y) = H_i(x)H_j(y)`.
• One can verify that plugging these for `B_(i,j)` in the equation of the last slide gives the bilinear interpolation formulas.

Generalizations and Edge Preservation

• We can choose basis functions with all kinds of shapes and degrees.
• In image editing tools, reconstruction if often done using some of bicubic basis functions.
• One problem with the simple constant and bilinear techniques we have discussed is that edges or lines in the reconstructed image tend to be blurred out.
• There are more advanced nonlinear techniques that attempt to maintain sharp edges even in reconstruction. The book gives some references for these techniques.

Resampling

• In the light of what we've learned about sampling and image reconstruction let's return to the problem of texture mapping
• In texture mapping, we start and end with a discrete image.
• The actual mapping we do involves both a reconstruction stage and a sampling stage
• We will look at techniques for doing these geared specifically to the texturing problem.

Ideal Resampling

• Suppose we start with a discrete image or texture `T[k][l]` and apply some 2D warp to this to obtain an output image `I[i][j]`.
• How should we set each of the output pixels?
• One possible sequence of steps to perform would be:
  • Reconstruct a continuous texture `T(x_t, y_t)` using a set of basis functions `B_(k,l)(x_t, y_t)`.
  • Apply the geometric warp based on triangle and camera geometry to the continuous image.
  • Integrate the result against a set of filters `F_(i,j)(x_w, y_w)` to obtain the discrete output image.
• Let `M(x_w, y_w)` be the geometric transform. Putting these steps together yields the equation:
  `I[i][j] leftarrow int int_Omega dx_w dy_w F_(i,j)(x_w,y_w)sum_(k,l) B_(k,l)[M(x_w, y_w)]T[k][l]`
  `=sum_(k,l)T[k][l]int int_Omega dx_w dy_w F_(i,j)(x_w,y_w)B_(k,l)[M(x_w, y_w)]`
  which tells us how each pixel can be obtained as a linear combination of the input texture pixels.
• We will use this as the starting point for our discussion of resampling next day.