ACM Logo  An ACM Publication  |  CONTRIBUTE  |  FOLLOW    

Putting learning systems to work

By Theresa Edgington / May 2005

Print Email
Comments Instapaper

An interesting convergence is occurring between knowledge management, learning systems, and project management. Traditionally, workers were educated prior to task assignment (i.e., they were expected to be competent prior to undertaking a task). On-the-job learning consisted mainly of increasing efficiency with respect to known skills. However, in today's cost-constrained economy, teams are increasingly formed due to their propensity to learn and adapt. It is not uncommon to observe that a substantial amount of knowledge must be created within one or more key team members, not prior to the project, but during it.

As technology investments receive heavy scrutiny, it becomes prudent to leverage investment technology such as e-learning into profit-driven applications. This technology has been recently applied to improving school systems [3]. Industry projects, too, can benefit by integrating: knowledge management for sharing and creating knowledge and artifacts; electronic learning systems to remove temporal and geographic limitations, and to create a structure conducive to learning;
and project management to meet the objectives of the intended project within time and budget constraints.

This case study discusses the results of one such learning project, undertaken at a leading semiconductor manufacturing organization in cooperation with a university research center. Of interest is how e-learning technology facilitated business process management. Also of interest is to note that active management of the technology is necessary; the introduction of the e-learning system was not initially embraced by the team. Readers in adaptive learning projects may find our taxonomy, recommendations, and lessons learned to be useful for their own situations.

Creating a Learning-Based Project Centered Tool

University research centers unify applied research and industry relevance for innovative results. Such centers aid industry to examine likely, yet unproven, processes and/or technologies. In such an environment, knowledge management is critical; the need to work within a defined schedule is paramount; and the ease of acquiring a plethora of specialized tools is paradoxically unlikely. Thus, these constraints motivated us to employ learning system technology within our project development efforts.


Our project was chartered to create an initial ontology that would contribute to cross-functional communication [2] for failure analysis and failure identification (FA/FI) within the semiconductor industry. The academic team consisted of a professor, an associate professor, and two Ph.D. students. The industry team included a quality manager, various lab analysts, managers, and selected customers of the microscopy lab where FA/FI analysis was conducted. The academic team and the industry quality manager became the core team; other team members participated opportunistically as the project evolved. A management steering committee of industry and academic directors met periodically to review results and status.

The core team had valuable, yet disparate backgrounds, yet had not worked together previously. The academic team was geographically distant from the industry quality manager. Additionally, the project itself had to occur in an accelerated fashion with little time available for education and team-building. Therefore, adopting collaboration and learning technology to address the aggressive, yet increasingly all-too-common project expectations, was imperative.


The roles involved are typical of many knowledge-oriented projects. In addition to the core team, who have direct responsibility for the project's results, are non-core team members who participate only with regard to knowledge transfer. They are not required to improve their own expertise nor are they required to produce the anticipated results for the project. It should be noted that while each member of the core team acts primarily in one of the roles defined below, such a small team becomes highly productive when participants act outside of their primary roles whenever such participation is collaboratively productive.

Functional Director: The functional director acts as the governance structure moderating project activities to external strategy and requirements, attending planning meetings and participating actively in interviewing sessions, but remaining primarily in a guidance and strategic planning role.

Project Owner: The project owner drives the project from an operational task and schedule perspective and performs coordination for meetings and interviews. The project owner actively participates in interviewing and project planning sessions.

Site Owner: The site owner constructs and directly manages the KM/learning/project management technology. These duties include initial population of the learning system (hereafter called "the site"), administration and organization of material, and site strategy to encourage site participation. Additionally, the site owner is an active team participant and not merely a support person.

Team Participant: The role of team participant, performed to some extent by all core team members, is the development of the project's results, which include participation in interviews, research and uploading of helpful support materials, development of presentation material and documentation of results, model construction, and assessment activities.

Results Owner: The results owner implements the project's results. Traditionally, this role can be considered as a passive customer, but optimally, the results owner is actively involved (which occurred in our project). The results owner, as the recipient of the implementation, is highly motivated to encourage its success. In our project, the results owner coordinated meetings and operational communication between the academic team and the non-core team members. Initially her critical responsibilities included assisting in the identification of domain vocabulary and process, and promoting credibility of the core team to the other project participants in terms of the importance of the project to their organization and in the anticipated benefits to the team participantsfrom the success of the project. By actively participating, the results owner becomes a first-order core team member and minimizes organizational problems.

Blackboard Typography Design

We employed the Blackboard Learning System, and while structured for traditional education environments (i.e., courses) [1], we used creativity in expanding it to a project management role. Some Blackboard menu options are constrained, so we adapted a reasonable labeling system (see Figure 1). Our selection of menu options created a knowledge resource center with knowledge management, learning, and project management capabilities. Each of the selections is described in this section. Announcements generally guide the usage of the site (and by default, the project); additional sections store reference material (Books, Web sites, Staff Information). The final sections cover direct materials and interaction relating to project objectives (Information, Projects, Communication, and Discussion Board). The Tools option (at this level) is standard, relating to the mechanical use of the Blackboard system.

For each section (i.e., topic category), an overview message describes the category's purpose and gives examples of appropriate, and, if necessary, inappropriate material. For inappropriate material, a more appropriate section is suggested in the overview, such as the Discussion Board being the proper area for interaction.


This section provides system coordination, announcing new additions to the resource center, as well as for general knowledge or guidelines. The Blackboard system does not automatically announce new material (with the exception of the Discussion section which does alert one to new messages). Additionally, no method indicates new, versus previously read, submissions, which is a bit tedious when using a system where a sequential or logically prescribed flow cannot be predetermined. Therefore, we used two mechanisms: email for priority notification and announcements for all newly entered messages and materials. With this plan, email briefly notes that new material exists on the site. (Email of new material encourages prompt participation; as interaction and new material increase on the site, over time the use of email is reduced only to interactions needing priority attention.) Announcements, then, provide detail on where the material can be found.

The Announcements section is also useful for reminders, after an initial message is posted in the appropriate Discussion topic (see below). Announcements are not encouraged for meeting announcements as daily, or regular, use of the system is not required.


The Books section provides peripheral information, i.e., information useful but not critical to the project. ("Books" and "Information" were chosen because of the limited number of options allowed in the first level menu system. As all other levels supported user-defined labels, it is only a minor inconvenience.) This separation (from directly related material, which we included in the Information section) reduces the volume of reference material anticipated. As such, this section includes books, articles, and related information. Whether the material is of peripheral or direct value is determined when submitting files.

Web Sites

Some material optimally lists web link references in the Web Sites section. This is useful to point to a body of knowledge and preserve intellectual property rights that might exist with certain materials.

Staff Information

The Staff Information includes normal contact information relating to the core team project members, such as name, title, email address, phone, and photo (optional).


The Tools section is a standard option allowing the vendor and our own university IT organization to provide links relating to procedural and mechanical use of the Blackboard system.

The next four sections of the site constitute those most directly associated with the project deliverable and its progress.


The Information section is for materials known to directly support the project but not created by the project. These materials often (as they did for us) relate to assumptions and justifications for project deliverables.


The Projects section includes all direct documents and materials developed. Project activities and materials are further sub-divided into Logistics, Deliverables, Sample Data, Context Information, and Tools. Logistics relate to meeting schedules, agendas, bridge numbers used for conferences, and meeting minutes. Deliverables relate to artifacts developed (e.g., in our case, for the FA/FI ontology model), including vocabulary, data flow diagrams, use cases, and reports. Sample Data relate to detailed reports, diagrams, and database models. Context Information includes diagrams and interview notes; this section is a work-in-progress section in contrast to the Deliverables section which targets final deliverables. The Tools section here is specific to technology anticipated to be used during the project.


The Communication section provides interactive communication allowing members to send email without leaving the site. It also includes a discussion board, which is also available as a first-level option.

Discussion Board

The Discussion Board supports interaction and is preferred over email for significant project dialogue. While email is ad hoc and unthreaded, the discussion board topics allow for linked interaction flow. This capability is most useful when adding participants once the site has become active for some time. It also provides a better memory tool than email in determining how previous decisions and conclusions occur.

Developing a Learning-Enhanced Project Development Process

Naiveté may allow one to believe that building this type of system guarantees participation. We did not formally analyze the barriers to participation (which existed), but we did identify certain processes that encourage an optimal level of participation.

Material Population

The first task for this site is to develop an initial set of useful materials. Within our team, every core team member was given storage rights and encouraged to add material to the appropriate areas. Members were not required to enter their material; they could forward it to the site owner. This arrangement should be carefully considered with respect to unique situations in every project. Having the site owner enter all materials (excluding Discussion Board interaction, of course) may create a backlog of materials to be entered if site management is not a dedicated job responsibility. On the other hand, it is not unusual for a team member to enter material in the wrong section. (The software does not allow for the transfer of materials from one section to another. The original material was deleted and then reentered as new material in the appropriate section.) Due to explicit instructions added to each section, we had few incorrect submissions. We did have team members who seldom submitted materials directly, preferring to request its inclusion by the site owner.

Direct submission identifies the contributor, so the site owner always explicitly acknowledged the source of material she submitted to the site. This action encouraged group ownership. The site was originally viewed as an archival reference resource, but we integrated it actively into project execution.


Our team members did not immediately embrace participation. Blackboard does allow the site owner to track site access by individual and to identify specific sections and time of day usage; this support and three events particularly led to a highly interactive site. The first occurred when the functional director began referring to site information in meetings. By referencing site materials, the functional director implied his support which became a strong influence to team members to view the material. By the second meeting when the functional director referenced site materials, everyone had accessed the main areas of the site at least once.

Secondly, the site owner initially provided critical material both by email and by submission to the site. Quickly, the critical material was subsequently only submitted to the site and email notification was gradually reduced to only the most critical of messages. These messages, using the same gradual progression, provided less detailed information and finally only acknowledged that there was new material on the site to view.

The third event was the most productive. This occurred as the results owner began participation. As the core team was largely from one formal group (academics) and the results owner was from our industry partner (and could be viewed as our customer), we limited initial participation until the site was considered a productive tool and not a superfluous one. Within a month of use, we added the results owner who enthusiastically supported it. Once the results owner became active in the discussion boards and forwarded documents for the site owner to submit, full core team interaction was established.

From tracking metrics provided by the system (see Figure 2), we obtained a view of site usage by individual and by category. We grouped the topics into six categories: New Info & Announcements (messages indicating new material added or other general project announcements), Project Planning (project schedules, minutes, and progress), Domain Scoping (interviews and vocabulary extraction from domain experts), Domain Questions (interactive dialog relating to domain details), a Segue section (project options for alternative processes and objectives), and Resource Center Use (guidelines and questions regarding the use of the system).

Figure 3 provides an interesting view of access by primary area taken at the end of the project. (Student and group areas are more appropriate to a teaching, versus a project, environment.) Initially expected to act merely as a reference site (materials submitted in the main content areas), the primary use of the site related to communication (57 percent), predominantly the discussion board. A synergistic effect occurred as documents referenced or drafted were reviewed and discussed in the same system.

Lessons Learned

The opportunistic nature of projects is common in today's industry, particularly for technology organizations. Schedules are painfully constrained with restrictive budgets. The following highlights a number of lessons learned from this experience.

Establish flexible roles: In our project, certain roles focused more on coordination and management while others focused more on task. However, even managers became workers for certain tasks, as appropriate. Further, the use of a learning system as a project management tool was a non-management decision—implemented by a team member who had been investigating learning technology.

Ensure system participation: We needed the core team to become active and highly interactive quickly. Communication for this type of project is equally, and arguably more important, than other types of productivity from individual members. Active support by the project's leadership is critical, allowing the system to function on a project level and to become more than a reference and archival mechanism.

Manage the system: We limited participation while the project was active to core team members. As the project was concluding, other specified individuals were given access to review the site's contents. We polled the team for their reactions to site membership expansion and allowed them to delete any of their own messages in the Discussion Board that they felt might be of a sensitive nature. It should be noted that site access expansion, particularly beyond the core team, should be made clear early to retain trust and long-term credibility.

Develop a workable taxonomy: We openly discussed how to organize various sections of the site. In fact, the results owner provided the best taxonomy for our Projects section. Using a "results" focus not only aligned the taxonomic structure of the documents, but indirectly aligned the cognitive focus of the team toward the practical use of the deliverables.

Approach the technology practically: The learning system technology was somewhat inflexible. We were fortunate not to have fully constructed our site organization before involving all participants. Moving messages and documents from one taxonomy to another is manually intensive. In one restructuring instance, we discontinued using an old category, but did not delete it. A final message was added stating the rationale for its demise and a pointer to where future messages could be found.

Provide informal metrics: We provided periodic assessment on the activity levels for the system. Such feedback encourages management to consider the site favorably (and to continue to support it), and is of at least mild interest to other team members.


As project schedules collapse and teams form dynamically, learning technology aids the project-management process. We did not measure the financial contribution of this technology, but because of it, we were able to eliminate travel planned for four core team members during the four-month project without any negative impact. In fact, capturing both materials and interaction improved our project results by allowing team members to go back and review past interactions, and in adding team participation intermittently. It notably contributed to producing novel results on schedule.


1. Blackboard Learning System Overview, Blackboard Inc., December 2002,

2. Edgington, T. M.; Choi, B-J.; Henson, K.; Raghu, T. S.; and Vinze, A.; "Ontology-enabled Knowledge Management: Adopting Ontology to Facilitate Knowledge Sharing," Communications of the ACM, November 2004.

3. Laffey, J. M.; Musser, D.; Remidez, H.; and Gottenker, J.; "Virtual Extension: Networked Systems for Schools that Learn," Communications of the ACM, 46, 9, (September 2003), 192-200.


  • There are no comments at this time.