Project: Affective Computing

Evaluation of a user's emotional experience with technology is not well understood, especially when the primary goal of a technology is to entertain (e.g., computer game) or to invoke an emotional experience (e.g., animated film). The goal of this project is to develop and validate a suite of reliable, valid, and robust methods for evaluating affective user experience in computer games, animations, and new media environments that address the unique challenges of these technologies. We build affective evaluation methods based on users' physiological signals, including their cardiac activity, galvanic skin response, brain activity, eye gaze and blink patterns, pupil diameter activity, and muscular response. In addition, we integrate less-invasive approaches, including facial expression image recognition, thermographic imaging, interactive behavioural measures, and subjective responses. Our models of user affect are built and validated in both laboratory and real-world settings to ensure both scientific relevance and applicability to our industry partners in computer games, new media, and animation.

In addition to understanding user experience in games, we use affective information to drive the interaction itself. For example, by combining affective or emotional input with traditional game input, we can create a compelling game experience that beats the willing suspension of disbelief into submission.

Participants

Regan Mandryk
University of Saskatchewan
Lennart Nacke
University of Ontario Institute of Technology, University of Saskatchewan
Michael Kalyn
University of Saskatchewan
Calvin Lough
University of Saskatchewan
Clayton Epp
University of Saskatchewan
Brett Taylor
University of Saskatchewan
Ian Livingston
Electronic Arts
Max Birk
University of Saskatchewan
Kristen Dergousoff

Publications

Creating and Interpreting Abstract Visualizations of Emotion
Taylor, B., Mandryk, R. (2012), Graphics Interface 2012, Toronto, ON. 61-68.
Emotional response and visual attention to non-photorealistic images
Mould, D., Mandryk, R., Li, H. (2012), Computers & Graphics, vol. 6 no. 6, 658-672. <doi:10.1016/j.cag.2012.03.039>
Evaluation of Emotional Response to Non-Photorealistic Images
Mandryk, R., Mould, D., Li, H. (2011), Non-Photorealistic Animation and Rendering, Vancouver, BC, Canada. 7-16.
Identifying Emotional States Using Keystroke Dynamics
Epp, C., Lippold, M., Mandryk, R. (2011), Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems (CHI 2011), Vancouver, BC, Canada. 715-724. <doi:10.1145/1978942.1979046>
Biofeedback Game Design: Using Direct and Indirect Physiological Control to Enhance Game Interaction
Nacke, L., Kalyn, M., Lough, C., Mandryk, R. (2011), Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems (CHI 2011), Vancouver, BC, Canada. 103-112. Honorable Mention Award. <doi:10.1145/1978942.1978958>
Influencing Experience: The Effects of Reading Game Reviews on Player Experience
Livingston, I., Nacke, L., Mandryk, R. (2011), 10th International Conference on Entertainment Computing (ICEC'11), 89-100.
The Impact of Negative Game Reviews and User Comments on Player Experience
Livingston, I., Nacke, L., Mandryk, R. (2011), Proceedings of the 2011 ACM SIGGRAPH Symposium on Video Games, Sandbox '11. Vancouver, BC, Canada. 25-29. <doi:10.1145/2018556.2018561>
Brain and Body Interfaces: Designing for Meaningful Interaction
Fairclough, S., Gilleade, K., Nacke, L., Mandryk, R. (2011), Proceedings of the 2011 Annual Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA'11), Vancouver, BC, Canada. 65-68. Workshop, more information at: http://brainandbody.physiologicalcomputing.net. <doi:10.1145/1979742.1979591>
The Neurobiology of Play
Bateman, C., Nacke, L. (2010), Proceedings of Future Play 2010, Vancouver, BC. 1-8. <doi:10.1145/1920778.1920780>
Developing a triangulation system for digital game events, observational video, and psychophysiological data to study emotional responses to a virtual character
Kivikangas, J., Nacke, L., Ravaja, N. (2011), Entertainment Computing, vol. 2 no. 1, 11-16. Video Games as Research Instruments (Special Issue). <doi:10.1016/j.entcom.2011.03.006>
Correlation between heart rate, electrodermal activity and player experience in First-Person Shooter games
Drachen, A., Nacke, L., Yannakakis, G., Lee Pedersen, A. (2010), Proceedings of the 5th ACM SIGGRAPH Symposium on Video Games, Los Angeles, CA. 49-54. <doi:10.1145/1836135.1836143>
Electroencephalographic Assessment of Player Experience: A Pilot Study in Affective Ludology
Nacke, L., Stellmach, S., Lindley, C. (2010), Simulation & Gaming, vol. 41 no. ahead of print, 1-24. <doi:10.1177/1046878110378140>
Brain, body and bytes: psychophysiological user interaction
Girouard, A., Solovey, E., Mandryk, R., Nacke, L., Tan, D., Jacob, R. (2010), Proceedings of the 28th of the international conference extended abstracts on Human factors in computing systems, CHI. Atlanta, GA. 4433-4436. Workshop. <doi:10.1145/1753846.1754167>
More Than a Feeling: Measurement of Sonic User Experience and Psychophysiology in a First-Person Shooter Game
Nacke, L., Grimshaw, M., Lindley, C. (2010), Interacting with Computers, vol. 22 no. 5, 336-343. Special Issue: Modelling user experience - An agenda for research and practice. <doi:10.1016/j.intcom.2010.04.005>
Wiimote vs. Controller: Electroencephalographic Measurement of Affective Gameplay Interaction
Nacke, L. (2010), Proceedings of Future Play 2010, Vancouver, BC. 159-166. <doi:10.1145/1920778.1920801>
Methods for Evaluating Gameplay Experience in a Serious Gaming Context
Nacke, L., Drachen, A., Goebel, S. (2010), International Journal of Computer Science in Sport, vol. 9 no. 2, Darmstadt, Germany. http://iacss.org/index.php?id=96.
Physiological Measures for Game Evaluation
Mandryk, R. (2008), Game Usability: Advice from the Experts for Advancing the Player Experience, Morgan Kaufmann. K. Isbister and N. Shaffer, Eds.
Affective Ludology: Scientific Measurement of User Experience in Interactive Entertainment
Nacke, L. (2009) Ph.D. Dissertation, Karlskrona, Sweden. Blekinge Institute of Technology. http://phd.acagamic.com, (ISBN) 978-91-7295-169-3.
A Fuzzy Physiological Approach for Continuously Modeling Emotion During Interaction with Play Environments
Mandryk, R., Atkins, M. (2007), International Journal of Human-Computer Studies, vol. 6 no. 4, 329-347. <doi:doi:10.1016/j.ijhcs.2006.11.011>
A Continuous and Objective Evaluation of Emotional Experience with Interactive Play Environments
Mandryk, R., Atkins, M., Inkpen, K. (2006), Conference on Human Factors in Computing Systems (CHI 2006), Montreal, Canada. 1027-1036. <doi:10.1145/1124772.1124926>
Using Psychophysiological Techniques to Measure User Experience with Entertainment Technologies
Mandryk, R., Inkpen, K., Calvert, T. (2006), Behaviour and Information Technology (Special Issue on User Experience, vol. 25 no. 2, 141-158. <doi:10.1080/01449290500331156>
A physiological approach for continuously modeling user emotion in interactive play environments
Mandryk, R. (2008), Proceedings of Measuring Behavior 2008, Maastricht, NE. 93-94.
Physiological Indicators for the Evaluation of Co-located Collaborative Play
Mandryk, R., Inkpen, K. (2004), Proceedings of CSCW, Chicago, USA. 102-111. <doi:10.1145/1031607.1031625>
LAIF: A Logging and Interaction Framework for Gaze-Based Interfaces in Virtual Entertainment Environments
Nacke, L., Stellmach, S., Sasse, D., Niesenhaus, J., Dachselt, R. (2010), Entertainment Computing, ahead of print. <doi:10.1016/j.entcom.2010.09.004>
How Mobile is Mobile Gaming? Contextual Influences on Mobile Player Experience - A Model Proposition
Engl, S., Nacke, L. (2010), Entertainment Computing, Duisburg, Germany. http://sunsite.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-634/.
Player-game Interaction Through Affective Sound
Nacke, L., Grimshaw, M. (2011), Game Sound Technology and Player Interaction: Concepts and Developments, ISBN 978-1616928285. Chapter 13. 264-285. Hershey, PA, USA: IGI Global Publishing.. <doi:10.4018/978-1-61692-828-5.ch013>
Gameplay experience testing with playability and usability surveys - An experimental pilot study
Nacke, L., Schild, J., Niesenhaus, J. (2010), Playability and player experience: Proceedings of the Fun and Games 2010 Workshop, NHTV Expertise Series 10. Leuven, Belgium. 31-45. Workshop paper.