Chris Pollett >
Students > [Bio] [Del1] [Del2] [Del3] |
CS298 ProposalTransmitting Avatar Emotions over the WebJing Yuan Michelle_Yuan@yaho.com Advisor: Dr. Chris Pollett Committee Members: Dr. Ho Kuen Ng, Dept. of Mathematics and Computer Science (Ng@cs.sjsu.edu), Dr. Sin-Min Lee, Dept. of Mathematics and Computer Science (lee@cs.sjsu.edu) Abstract:Over the years psychologists have developed a number of encoding schemes for recording human facial expressions and to record hand gestures [E41], [EF69], [EF78], [M92]. Recently, there have been a number of AI projects, which attempt to use these recording schemes to write programs, which can recognize human emotions. The early example in development of facial animation technology began in 1992 by the Cambridge Research Laboratory and resulted in DECface [W1]. Several books and conference papers on Facial Animation research have appearanced [WL94A], [WL94B], [WL95], [FK96], [WLM98], [W92]. Imaginative applications of animated graphical faces are found in sophisticated human-computer interfaces, interactive games, multimedia titles, and, as always, in a broad variety of production animations. There has also been some work on an XML-based language, HumanML, for encoding such emotional and gesture information. Zong, Dohi, and Ishizuka [ZHMI00] have proposed an XML-based language MPML (multimodal presentation markup language), which is SMIL compliant for giving character agents emotions while they present things like PowerPoint slides. W3C organization also provides X3D that is XML based language for displaying 3D graphic. In this project, we propose to develop a network poker game with 3D-characters being able to express emotional information based on their cards. The technologies we will use for our project are XML, X3D, ECMAscript, and XSLT. Each client chooses 3D characters for the various players from a selection of characters available on his/her local machine, and, via a form associated player.xml files for those 3D characters are uploaded to the server. This uploaded file contains details about which emotions these 3D players support. The game then starts. The server receives emotion messages sent by each client (these are coded in an XML language of our design) and sends the message to the other clients that whose 3D character for that player have the ability to display the emotion specified by the message. Message passing will all be done using HTTP, using tricks with frames. A hidden send frame is used to send GET requests to the server with an emotion message and a hidden receive frame is used to poll the server periodically via the HTTP-REFRESH technique to get information about other clients' emotions. When a client receives an emotion message, it is used to apply a XSLT transformation to the player the emotion is supposed to appear on to make the emotion appear. CS297 Results
Proposed Schedule
Key Deliverables:
Innovations and ChallengesWe are proposing two new DTDs for emotions, which we hope might be useful in network gaming. Using XSLT is a novel approach to applying emotions to X3D characters. It is also challenging to get such transformations to work. Our approach uses existing open standards and might provide a new way to do network games that avoids proprietary systems such as DirectPlay (part of DirectX) from Microsoft. The technique of message passing in HTTP we intend to use is also novel and difficult to implement. References:1. [E41] Efron, D. Gesture and Environment. King's Crown Press. 1941. 2. [EF69] Ekman, P. and Friesen, W.V. The repertoire of nonverbal behavioural categories: Origins, usage, and coding. Semiotica 1:49-98. 3. [EF78] Ekman, P. & Friesen, W. V. The Facial Action Coding System (FACS): A technique for measurement of facial action. Consulting Psychologists Press. 1978. 4. [M92] David McNeill. Hand and Mind. University of Chicago Press. 1992. [W3C97] Extensible Markup Language (XML). www.w3.org/XML. 5. [W3C99] XSL Transformations (XSLT) Version 1.0. www.w3.org/TR/xslt. 6. [ZHMI00] Zong, Y., Hiroshi, D. and Mitsuru, I. Emotion expression functions Attached to Multimodal Presentation Markup Language MPML. 4th International Conference on Autonomous Agents. 2000. 7. [ECMA99] Standard ECMA-262 ECMAScript Language Specification 3rd ed. http://www.ecma.ch/ecma1/stand/ecma-262.htm. ECMA. 1999. 8. [W1] http://crl.research.compaq.com/projects/facial/facial.html 9. [WL94A] Waters, K. and Levergood, T. DECface: An automatic Lip-Synchronization Algorithm for Synthetic Faces, CRL Technical Report 93/4, September 1994 10. [WL94B] Waters, K. and Levergood, T. An Automatic Lip-Synchronization Algorithm for Synthetic Faces, ACM Multimedia, San Francisco, pages 149-156, October 1994 11. [WL95] Waters, K., Levergood, T. DECface: A System for Synthetic Face Applications, Journal of Multimedia Tools and Applications, Kluwer Academic Publishers, Vol. 1, No. 4, pages 349-366. Nov. 1995 12. [FK96] Computer Facial Animation, Fred I. Parke and Keith Waters. A. Peters Ltd., Boston Massachusetts, ISBN 1-56881-014-8 Hardcover, pp. 450, 1996. [software available for download] 13. [WLM98] Waters, K, Rehg, J, Loughlin, M, Kang, S, Computer Vision for Human-Machine Interaction, Roberto, C, and Pentland, A (eds)., Cambridge University press, pages 83-96, 1998 14. [W92] Waters, K. Modeling Three-Dimensional Facial Expressions, Processing Images of Faces, Bruce and Burton eds., Ablex, New Jersey, pages 202-227, 1992 |