Unit Testing

Definitions

In a sense, every method implements a mathematical function. The question is, does the method implement the intended function? A test case associates a method with a finite list of inputs and outputs from the function the method is supposed to implement. This list is called an oracle. Running a test case feeds the oracle's  inputs to the method, one at a time, then compares the outputs produced by the method with the outputs specified by the oracle. If they all match, then the method passes the test case, otherwise it fails.

For example, suppose we define a method that computes the length of a two-dimensional vector:

double length(Vec2d vec) {
   return sqrt(vec.xc * vec.xc + vec.yc + vec.yc);
}

Here's an example of an oracle for this method:

{[(1, 0), 1], [(0, -1), 1], [(1, 1), 1.414], etc.}

A test case is an object that encapsulates an oracle and a flag that is set by a run method:

class LengthTest extends TestCase {
   boolean pass = true;
   static int CAP = 100;
   Vec2d[] vecs = new Vec2d[CAP];
   double[] lengths = new double[CAP];
   int size = 0;
   void add(Vec2d vec, double length) {
      if (size < CAP) {
         vecs[size] = vec;
         lengths[size++] = length;
      }
   }
   void run() {
      for(int i = 0; i < size; i++)
         pass = pass && (length(vecs[i]) == lengths[i]);
   }
}

A test suite is a collection of tests. These tests may be test cases or other test suites. We can use the Composite Design Pattern to represent the situation:

Running a test suite means running each of its members. If any member fails, then the test suite fails.

Typically, we create test cases for each method in a class. We can collect all of these test cases together into a suite that tests all of the methods in the class. These class test suites can be collected together into a package test suite, and the package test suites can be collected together into a system test suite:

JUnit

JUnit is a freely downloadable framework for running Java test suites:

Example

Assume we are building a geometric modeling system (GMS). Such systems might form the kernel of a CAD/CAM system. Currently, GMS consists of thee subpackages:

The algebra package contains classes for representing vectors and matrices. The geom package contains classes for representing points, lines, and polygons. The graphics package contains classes for representing pens, erasers, and canvases.

Vec2d

For example, the algebra package contains a class for representing two-dimensional vectors:

package gms.algebra;

public class Vec2d {
   private double xc, yc;
   public Vec2d(double xc, double yc) {
      this.xc = xc;
      this.yc = yc;
   }
   public Vec2d() { xc = yc = 0; }
   double getXC() { return xc; }
   double getYC() { return yc; }
   // add this vector to another:
   Vec2d add(Vec2d other) {
      return new Vec2d(xc + other.getXC(), yc + other.getYC());
   }
   // computes dot product of this vector with another:
   double dot(Vec2d other) {
      return xc * other.getXC() + yc * other.getYC();
   }
   // computes the length of this vector:
   double length() {
      return Math.sqrt(dot(this));
   }
   // etc.
}

In the same package we add a test suite for this class:

package gms.algebra;

import junit.framework.*;
import junit.extensions.*;

public class Vec2dTest extends TestCase {
   // test fixture:
   static final int CAP = 5;
   Vec2d[] vec = new Vec2d[CAP];
   double[] length = new double[CAP];
   double delta = 1e-4; // = error tolerance

   // init test fixture:
   protected void setUp() { ... }

   // the tests:
   public void testLengths() { ... }
   public void testDots() { ... }
   public void testSums() { ... }

   public static Test suite() {
      return new TestSuite(Vec2dTest.class);
   }
}

The fields at the beginning of the declaration are called the test fixture. The fixture declares an array of vectors to be tested. It might also create several oracles. After an instance of the Vec2dTest class is created, the framework will automatically call the setUp method, which, like a constructor, will initialize the test fixture.

Each method in the Vec2d class has a corresponding test method in the Vec2dTest class. The test method must begin with the prefix "test". In each instance of the Vec2dTest class, one of these test methods will be the selected test. (This is called the Pluggable Selector Pattern.)

Notice the static suite method, which creates and returns a test suite. The framework will automatically call this method, then run the suite of tests it returns. The TestSuite constructor takes Vec2dTest.class as its input. Using reflection, the constructor will create one test case for each test method, then add the test case to its member list.

JUnit is a little confusing. Although Vec2dTest extends the TestCase class, each instance will have a selected test method as the test it will run. The test suite associated with the Vec2d class will consist of three Vec2dTest objects, one with testLengths as its selected test, one with testDots as its selected test, and the third with testSums as its selected test.

Our Vec2dTest class inherits a bunch of assertion methods that updates a test result object.

   public void testLengths() {
      for(int i = 0; i < CAP; i++) {
         assertEquals(vec[i].length(), length[i], delta);
      }
   }

   public void testDots() {
      assertEquals(vec[0].dot(vec[1]), 0, delta);
      assertEquals(vec[1].dot(vec[2]), 0, delta);
      assertEquals(vec[3].dot(vec[4]), 50, delta);
   }

   public void testSums() {
      Vec2d[] sum = new Vec2d[CAP];
      sum[0] = vec[1].add(vec[2]);
      sum[1] = vec[3].add(vec[4]);
      assertEquals(sum[0].getXC(), 1, delta);
      assertEquals(sum[0].getYC(), -1, delta);
      assertEquals(sum[1].getXC(), 100.1, delta);
      assertEquals(sum[1].getYC(), 200.2, delta);
   }

Here's a screen shot of JUnit after running the test suite:

Mat2x2

The Mat2x2 class represents two-by-two matrices:

public class Mat2x2 {
   private Vec2d[] row = new Vec2d[2];
   public Mat2x2(Vec2d row1, Vec2d row2) {
      row[0] = row1;
      row[1] = row2;
   }
   public Mat2x2() {
      row[0] = new Vec2d();
      row[1] = new Vec2d();
   }
   // gets row r column c entry of this matrix:
   public double getEntry(int r, int c) throws Exception {
      if (r != 0 && r != 1) throw new Exception("row out of range");
      if (c != 0 && c != 1) throw new Exception("col out of range");
      if (c == 0) return row[r].getXC();
      return row[r].getYC();
   }
   // multiples this matrix with a vector:
   public Vec2d mul(Vec2d vec) {
      return new Vec2d(row[0].dot(vec), row[1].dot(vec));
   }
   // computes the determinant of this matrix:
   public double det() {
      double axis1 = row[0].getXC() * row[1].getYC();
      double axis2 = row[1].getXC() * row[0].getYC();
      return axis1 - axis2;
   }
   // etc.
}

Here's are the corresponding test cases:

public class Mat2x2Test extends TestCase {
   // test fixture:
   static final int CAP = 5;
   Mat2x2[] mat = new Mat2x2[CAP];
   Vec2d[] vec = new Vec2d[CAP];
   double delta = 1e-4; // = error tolerance
   // init test fixture:
   protected void setUp() {
      mat[0] = new Mat2x2();
      mat[1] = new Mat2x2(new Vec2d(1, 0), new Vec2d(0, 1));
      mat[2] = new Mat2x2(new Vec2d(3, 4), new Vec2d(2, 5));
      mat[3] = new Mat2x2(new Vec2d(0, -1), new Vec2d(1, 0));
      mat[4] = new Mat2x2(new Vec2d(2, 2), new Vec2d(2, 2));
   }
   // the tests:
   public void testGetEntry() {
      try {
         assertEquals(mat[2].getEntry(0, 0), 3, delta);
         assertEquals(mat[2].getEntry(1, 0), 2, delta);
         assertEquals(mat[2].getEntry(0, 1), 4, delta);
         assertEquals(mat[2].getEntry(1, 1), 5, delta);
         assertEquals(mat[2].getEntry(2, 1), 5, delta);
      } catch(Exception e) {
         assertEquals(e.getMessage(), "row out of range");
      }
   }
   public void testDet() {
      assertEquals(mat[0].det(), 0, delta);
      assertEquals(mat[1].det(), 1, delta);
      assertEquals(mat[2].det(), 7, delta);
      assertEquals(mat[4].det(), 0, delta);
   }
   public void testMul() {
      Vec2d vec = new Vec2d(3, 7);
      Vec2d result = mat[2].mul(vec);
      assertEquals(result.getXC(), 37, delta);
      assertEquals(result.getYC(), 41, delta);
   }
   public static Test suite() {
      return new TestSuite(Mat2x2Test.class);
   }
}

Package Test

We add a package level test to the package

public class AlgebraTest extends TestSuite {
   public static Test suite() {
      TestSuite suite = new TestSuite();
      suite.addTestSuite(Vec2dTest.class);
      suite.addTestSuite(Mat2x2Test.class);
      // etc.
      return suite;
   }
}

System Test

Here's our system level test suite:

package gms;
import junit.framework.*;
import junit.extensions.*;

public class GMSTest extends TestSuite {
   public static Test suite() {
      TestSuite suite = new TestSuite();
      suite.addTest(gms.algebra.AlgebraTest.suite());
      suite.addTest(gms.geom.GeomTest.suite());
      suite.addTest(gms.graphics.GraphicsTest.suite());
      return suite;
   }
}

Here's a screen shot of JUnit after running this test suite. Note that one of the tests has failed.

Warnings

It's difficult to test graphics and user interface code.