Preliminary heuristics for the design and evaluation of online communities of practice systems

Online communities…communities of practice…knowledge management…human capital. For some time now, companies have believed that they can be more effective—and more profitable—if they could only get their people to talk to each other and share what they know. Globalization, acquisitions, and diversification have led to workforces that are more spread out and more varied. In addition to creating the need for a flexible, agile workforce, the evolving and sometime volatile business climate has reduced the ability of companies to rely on stable groups of people who slowly grow their expertise over many years.

To address this need, companies have built or bought knowledge management portals, tools for growing communities of practice, and other expensive, often complex software systems. As with other software systems, however, utilization is not guaranteed, and the return on investment may never materialize. What functionality should such systems provide to increase the probability of payback? How can product specifications be evaluated? By what benchmarks can existing implementations be inspected?

This article offers a set of heuristics drawn from published academic research into online communities of practice. These heuristics may be used to inform design, or they can be used as heuristics in an evaluation process. These heuristics are preliminary—we have successfully applied them informally to evaluate an existing system at a large corporation. During this evaluation, the issues uncovered matched well with the issues raised by users of the system. The heuristics also uncovered additional issues previously unrecognized. At the same time, we recognize that more testing needs to be conducted.

The Heuristics

Software systems supporting online communities of practice should consider providing the following functionality:

1. Support for ideation and the evaluation of brainstormed alternative

   An important aspect of communities is the ability to communally generate and evaluate new ideas about the processes within the practice as well as the function of the community. Tools that can provide this ability range from simple asynchronous discussion and synchronous chat to sophisticated packages designed specifically for idea generation, ranking, and selection of those ideas. Some key questions to consider for this dimension are:

   • Are there tools that enable creating a shared "virtual mental space"?
   • Do the tools allow the ranking and evaluating of generated alternatives?
   • Are there both synchronous and asynchronous mechanisms for participating in idea generation and evaluation?
   • Do all the tools allow equal participation?
2. Structured information and interaction

   Participants will more actively engage in electronic collaboration if the interactions are structured in time, space, and scope. Community in general is most effective when activity is driven by context of its rituals and its purpose. Tools, events, and content should be structured to provide this context for action. Specifically, some key questions concerning structure are:

   • Does the environment foster content-related interactions built around structured tasks?
   • Do the synchronous and asynchronous tools provide a structure for participation?
   • Are a variety of roles supported in the environment and the interactions (such as playing devil's advocate, synthesizer of ideas, or divergent thinker)?
   • Is there a "cyclical" nature to the events that take place in the space (regular meetings, rituals such as birthday recognition, rites of passage, etc.)?
   • Does the purpose of the interaction drive the design of the tools?
   • Do the tools allow creation of rich context (expert feedback, private reflection, debate, dialog tracking, etc.)?
3. Unifying purpose and focus

   A community thrives on the ability of its members to believe in and articulate its purpose and goals. If a community centers around a particular practice (such as a profession or use of a software package), it is important that members are able to use a common language in expressing what they are about and what they are trying to accomplish both to each other and to those outside the community. It is also important that they are able to address common problems that they are facing together and collaboratively. The following questions address this unifying purpose and focus:

   • Does the space allow the creation and support the articulation of a common and clear purpose (a charter or prominently figured statement of purpose, for example)?
   • Does the space allow for sub-communities that address the common purpose of the community as a whole?
   • Does the space provide for meaningful discussion about the definition of key terms and concepts?
   • Do the tools provide for discussions about cases that represent common problems or issues?
4. Dynamic information and structure

   Communities constantly evolve. Both the information and the people that participate in them are constantly in a state of flux. The space needs the ability to add, change, or delete information or functionality in response to community changes. Some key questions that should be asked are:

   • Are the "gathering places" extensible and do they allow an ability to grow both in size and in functionality over time?
   • Is each member able to update a personal profile of their expertise, interests, or other personal characteristics?
   • Does the space allow the creation of new subgroups as the context changes?
   • Does the space contain an easily-maintained and dynamic "storehouse" of documents, conversations, and other information?
5. Individual and group identity construction and maintenance

   Identity is critical to a community. Communities and the individuals in them must have vehicles for expressing how they are unique, what role they play in a larger scheme, and what maintains the bonds of the members. Identity is established through the roles of the members, the rituals of the group, and the history that they create together. Many of the questions addressed in the other dimension also apply to the establishing of identity:

   • Is each member able to update a personal profile of their expertise, interests, or other personal characteristics?
   • Is there a "cyclical" nature to the events that take place in the space (regular meetings, rituals such as birthday recognition, rites of passage, etc.)?
   • Does the space contain an easily-maintained and dynamic "storehouse" of documents, conversations, and other information?
   • Does the space allow for sub-communities that address the common purpose of the community as a whole?
   • Does the environment support the creation of social relationships?
   • Are items in the environment attributed to the people who created them?
6. Oversight—an appropriate level of control and moderation

   A community should allow for an adequate level of control and moderation. The ability to provide feedback about members' participation in dialog and other activities enables the environment to serve as a tool for instilling and maintaining the social norms of the group. The following questions should be considered:

   • Is there a mechanism for providing both public and private feedback to members?
   • Are "social rules" clearly articulated and readily available?
   • Does the environment provide a mechanism for continuous coaching of community behaviors?
7. A rich set of discussion tools, including support for dialog, negotiation, and collaborative problem-solving

   Tools in the environment should not only be structured around a particular purpose but also be rich and varied. A variety of different types of dialog take place in a community: negotiation, problem-solving, sharing, debate, and socializing, for example. Every effort should be made to limit members' feelings that the tools in the environment limit their communication styles. Specifically, the following questions should be asked:

   • Do the solutions provide for a variety of different communication styles including social conversation and issue-based discussion (debate, negotiation, dialog, problem-solving)?
   • Does the environment facilitate the sharing and negotiation of knowledge?
   • Do the tools permit members to seek quick clarification through questions?
   • Does the environment support both dyadic (one-on-one) as well as group interactions?
   • Can users interact around any information component of the system (document, profile, discussion, event)?
8. Mechanisms for regular stimulation toward increased growth and learning

   Communities evolve and grow over time, and the tools they use must not only accommodate this change but foster it as well. Learning is the key mechanism by which the community grows and evolves. Many of the questions listed above help promote a learning environment. Specifically, the following questions are important to consider:

   • Does the space support and stimulate collaborative problem-solving?
   • Does the environment allow experts to provide rich and timely feedback during discussions, learning events, and other interactions?
   • Does the environment provide tools for automated instigation of growth through timely messages, quotations, and questions?
9. Effective connections to live, offline community activities

   Communities generally have both online and offline activities. Often, communities will hold get-togethers, members will network in person, or collaborate on problems outside the context of the electronic space. The following questions should be considered in light of this other aspect of community:

   • Does the space enable the recognition and communication of offline events?
   • Do the tools enhance offline activities?
   • Is there a tool in the environment that allows members to identify and negotiate the availability of members for in-person meetings (awareness tools, calendaring, etc.)?

We hope that by offering these heuristics in preliminary form, we may generate discussion and other attempts to use, validate, or improve these principles. We also, in the near future, plan to publish an expanded version of the heuristics with references to all the sources from which they have been derived.