Publication: A Survey of Groupware Evaluations

Groupware software is still not widely used in spite of continued hardware advances in processing power and network speeds. This indicates that developers may not be matching the software they design with the needs of the target user groups. For this reason, a further understanding of groupware evaluation should be sought to help provide a means for identifying the key issues developers must address for successful implementations.

This paper presents the findings of a survey of groupware evaluations in CSCW Proceedings ‘90, ‘92, ‘94, ‘96, and ‘98. Articles were selected if they included evaluations of groupware applications or included thorough discussions of groupware applications. In order for an application to be classified as groupware, it had to allow multiple users to work collaboratively toward a common goal [23].

A total of 45 articles were included in the study. In order to collect a broad range of information, a set of categories and criteria were developed to serve as a collection framework for the data in the articles. Data in the evaluations found in the articles was then gathered and analyzed.

Of the 32 articles that included evaluations, only 41% evaluated complete implementations of groupware applications in real world settings, and even fewer of these evaluated the social, organizational, or cultural impact the software had on the work settings. Additionally, a high proportion of articles introduced new groupware applications but failed to discuss evaluation or included incomplete information. This paper discusses these findings and makes recommendations for improvements in current research methods and proposes new directions for research on groupware evaluation.




David Pinelle
College of Medicine, University of Saskatchewan
Carl Gutwin
University of Saskatchewan


Evaluating Groupware Usability
Many organizations, particularly in the design and manufacturing sectors, are becoming increasingly distributed.


Pinelle, D., Gutwin, C. 2000. A Survey of Groupware Evaluations., Technical Report HCI-00-01, Department of Computer Science, University of Saskatchewan.


@techreport {eval-techreport,
author= {David Pinelle and Carl Gutwin},
title= {A Survey of Groupware Evaluations},
year= {2000},
number= {HCI-00-01},
school= {Department of Computer Science, University of Saskatchewan}