CS298 Proposal

Abstract:

In this project, we use the eigenface technique [Turk91] to produce age-enhanced face images of missing children. The eigenface approach is based on using PCA (Principal Components Analysis) on image data. PCA allows orthogonal vectors (eigenvectors) to be computed as a new coordinate system for multi-variate data. The projection of the data onto the eigenvector with the greatest eigenvalue (principal component axis) results in maximum variance about that axis. For the remaining eigenvectors, decreasing eigenvalues correspond to decreasing variances of the projected data. The lesser eigenvectors that do not result in much scatter may be truncated without sacrificing accuracy of data reconstruction. PCA has the property such that the projected data results in a set of weights that represent linear proportions of the principal components. Thus high-dimensionality data is reduced significantly in dimensions, allowing efficient algorithms that use only the required eigenvectors for the largest few corresponding eigenvectors. The weights may then be used to reconstruct each original data point. If a set of weights is used to reconstruct a data point, such that the data point is one that the PCA classifier is not trained on, then the classifier attempts to reconstruct the data point closest to one of the training data points.

The idea is to implement software that takes color images of faces as input and allows users to select and remove specified features of the images (e.g. eyes, nose, and mouth) to be PCA trained. Prior to this, the images are preprocessed in terms of size (all images of same size), rotation (removal of in-plane rotation), and cropping (eliminate background and maximize face space). For each selected feature, each color (RGB) channel is extracted to produce three separate grayscale images that are concatenated to form an overall grayscale image. This is done for original and age-enhanced sets of images, results of which are also concatenated to form final training grayscale images for each feature. These training images provide a mapping between original and corresponding age-enhanced features. The face images with the selected features removed are also PCA trained. An input face image is preprocessed in a similar manner as mentioned above except that the selected features are concatenated with themselves. Each selected feature from the input image is then projected onto a corresponding PCA trained feature space to produce a set of weights, which are then used to reconstruct a predicted aged feature; this is also performed on the feature-removed face images. The aged features are then blended back into the age-predicted feature-removed faces at the same locations.

CS297 Results

• Verified the eigenface approach by implementing a program that trains a PCA classifier to successfully recognize and reconstruct preprocessed digital images of missing children
• Implemented three Python scripts to support the downloading and preprocessing (image resizing, rotation, color format, and cropping) of digital images of missing children from a specific website (www.missingkids.com)
• Implemented a program to verify the feasibility of using shape contexts [Belongie01] and weighted bipartite matching [Jonker87] to locate features on a face image
• Further validated the eigenface approach by successfully reconstructing age-enhanced images using concatenated training images of original and aged images

Proposed Schedule

Key Deliverables:

• Software
  • Allows user to train program to reconstruct age-enhanced images via a features-based approach
  • Allows user-selected features on a template image to be automatically extracted from training and input images
  • Relies on and ties in much of the code from CS297
    • Downloading of digital images of missing children
    • Preprocessing of digital images to be used by program
    • Face recognition and reconstruction using eigenfaces
• Report
  • Goals, results, and assumptions of project
    • Demonstration of successful age-enhancement / reconstruction of known and unknown input images
    • Theoretical and practical limitations
  • Theory
    • Eigenfaces
    • Principal components analysis
  • Image Data
    • Image acquisition methods
    • Image preprocessing techniques
    • Image post-processing (image clean-up, filtering, etc.)
  • Software architecture and operations

Innovations and Challenges

• Well designed user interface that supports image loading, feature selection, and display of age-enhanced / reconstructed images
• Initially rely on the preprocessed alignment of the training images for locating user-selected features; must research other alternatives to achieve feature location on training and input images if results are not satisfactory
• May need to due some image post-processing to clean up age-enhanced / reconstructed image, due to the fact that the aged features may have different pixel intensities when blended back into the corresponding faces

References:

[Belongie01] S. Belongie, J. Malik, and J. Puzicha. Shape matching and object recognition using shape contexts. Technical Report UCB//CSD-00-1128, UC Berkeley, January 2001.

[Bishop 95] Neural Networks for Pattern Recognition. Christopher M. Bishop. Oxford University Press, 1995.

[Duda01] Pattern Classification. Richard O. Duda, Peter E. Hart, and David G. Stork. John Wiley & Sons, 2001.

[Gonzales02] Digital Image Processing. Rafael Gonzalez and Richard E. Woods. Prentice Hall, 2002.

[Jonker87] R. Jonker and A. Volgenant. "A Shortest Augmenting Path Algorithm for Dense and Sparse Linear Assignment Problems", Computing 38, 325-340, 1987.

[Press88] Numerical Recipes in C. William H. Press, Brian P. Flannery, Saul A. Teukolsky, William T. Vetterling. Cambridge University Press, 1988.

[Russell03] Stuart Russell and Peter Norvig. Artificial Intelligence: A Modern Approach. Pearson Education, Inc., 2003.

[Seul00] Practical Algorithms for Image Analysis: Description, Examples, and Code. Michael Seul, Lawrence O'Gorman, and Michael J. Sammon. Cambridge University Press, 2000.

[Smith02] A Tutorial on Principal Components Analysis. Lindsay I. Smith. http://www.cs.otago.ac.nz/cosc453/student_tutorials/principal_components.pdf, 2002.

[Turk91] Eigenfaces for Recognition. Matthew Turk and Alex Pentland. Journal of Cognitive Neuroscience, Vol. 3, No. 1, 1991.