Advisor: Dr. Chris Pollett

Description: Over the years psychologists have developed a number of encoding schemes for recording human facial expressions and to record hand gestures [E41], [EF69], [EF78], [M92]. Recently, there have been a number of AI projects which attempt to use these recording schemes to write programs which can recognize human emotions. There has also been some work on an XML-based language, HumanML, for encoding such emotional and gestural information. Zong, Dohi, and Ishizuka [ZDI00] have proposed an XML-based language MPML (multimodal presentation markup language) which is SMIL compliant for giving character agents emotions while they present things like PowerPoint slides. The goal of my master's will be develop an XML-based language for character avatars to which emotional and gestural information can be readily applied. The idea would be that a stylesheet could be applied to a character specified in my language to generate an 3D-object in a language like X3D, then gesture messages could be applied to this object with an additional transformation (which is not character specific). To test out this set-up in the context of on-line gaming, I will develop a network poker game with 3D-characters being able to express emotional information based on their cards.

Schedule:

Deliverables:

1. Become reasonably proficient at X3D/VRML. This will be demonstrated by creating a 3D mock-up of my appartment.

2. Create a DTD for ShML a language given in Dr. Pollett's CS146 Fall semester HW2. Write a XSL transform to render this language in X3D (as a flat 3D object).

3. First pass at XML character language complete.

4. Prepare some stylesheets to generate one character agent done. Test of applying emotion messages to character.

5. First semester report on project of length between 10-20 pages.

References:

[ANM97] VRML 2.0 Sourcebook. 2nd ed. A. Ames, D. Nadeau, J. Moreland. Wiley. 1997.

[E41] Efron, D. Gesture and Environment. King's Crown Press. 1941.

[EF69] Ekman, P. and Friesen, W.V. The repertoire of nonverbal behavioural categories: Origins, usage, and coding. Semiotica 1:49-98.

[EF78] Ekman, P. & Friesen, W. V. The Facial Action Coding System (FACS): A technique for measurement of facial action. Consulting Psychologists Press. 1978.

[M92] David McNeill. Hand and Mind. University of Chicago Press. 1992.