ACM Logo  An ACM Publication  |  CONTRIBUTE  |  FOLLOW    

What's important in a learning content management system

By Michael Feldstein / May 2002

Print Email
Comments Instapaper

As trainers and other educators suddenly find themselves evaluating expensive and complex server-side software—software in a functionality category that didn't even exist two years ago—it's hard to know what's what. To begin with, evaluating enterprise-class applications is inherently hard. Even the IT professionals aren't always very good at it. To make matters worse, the tools are constantly evolving and previously distinct software categories are merging as vendors rush to cram every feature imaginable into their offerings in an effort to stand out in the crowd. This article is my attempt to help you sort through at least some of the noise by helping you to answer one important question:

What do you really need?

It's easy to figure out which product has the most features. Much harder is figuring out which one has the right features. As somebody who has seen clients have both wonderful and terrible experiences with their LCMS's, I'm going to do my best to distinguish which features and which implementations of those features tend to give the best return-on-investment.

This article will not be a complete guide to everything you need to know about an LCMS. I don't know everything that you need to know about an LCMS because I don't know your particular situation. Nor am I intimately familiar with even half of the LCMS products on the market today. I won't provide a laundry list of all the features these various products have and I also won't discuss any of the features that I don't consider to be directly related to the management of learning content (or even some that are related, like personalization of content delivery.)

I'm going to focus on the problems of learning content production, since it is the content production problem that the CMS was invented to solve. Since organizations that purchase LCMS's typically start looking at these tools because they are faced with the challenge of producing and maintaining a daunting amount of custom e-learning content, I'm going to focus on the common problems that make that challenge so difficult and the low-hanging fruit where a properly designed LCMS can dramatically increase efficiencies, reduce cost, and release more time and resources to focus on the what really matters: teaching learners.

The rest of this article is organized into the following sections:

  • The History of Content Management
  • What's Unique About Learning Content
  • The Need
  • Templating and Navigation Automation
  • Workflow Automation
  • Bad Ideas and Other Distractions
  • What I'd Like To See Next

The History of Content Management

As far as I know, the modern content management system was invented in 1996 by c|net, the Internet news publishing company. As one of the web's first high-volume publishers, c|net faced some fairly serious production challenges that were new to the world. Web publishing is a fairly complex process. First, the author has to submit an article, written using tools that are intuitive for a non-technical writer to use. Then an editor needs to review the article and make changes, possibly even sending the article back to the author for editing. Next, it needs to go on to the copyeditor. Multiple versions of the document could be flying back and forth among the author, editor, and copyeditor. Once the content is finalized, it needs to be put into the layout in which it will appear on the site. The editor may decide to add related links. Somebody needs to decide where on the site the article will go. And once it's all done, somebody has to post the article to the public web site.

This is a challenging process to manage with even a handful of articles. C|net had to manage it with dozens of articles every day. Plus, with the creation of "Internet time," news needed to be updated even faster than daily newspapers had ever had to respond. The company needed to be able to collapse the entire process for publishing an article down to a matter of hours while simultaneously handling a huge number of articles.

The solution to this kind of problem is to automate any part of the process that doesn't require human judgment. Writers can type their articles directly into the publishing system using a web-based text editor. When the article is submitted online, it can instantly be made available to the editor, who also could receive an email alert that a new article is ready for review. The editor can then make changes online and click the "submit" button, which sends it on to the copy-editor. Since most articles are generally presented in one of a small handful of well-established layouts, the editor need only choose the appropriate layout template and the article will automatically be formatted appropriately. Likewise, the editor can click a few buttons to choose where on the site the article will be published and when it will go up, and the system will take care of the rest. No further human intervention is necessary. (Note that I'm not familiar with c|net's particular publication workflow. What I'm describing here are relatively generic features of a CMS.)

In this new system, most of the painful issues of paper passing, logistics, and scheduling are handled by the software. Editors can focus on editing. Writers can focus on writing. Programmers and graphic designers aren't really needed for the production of typical articles, and when they are needed they are only handling the aspects of the article that are unique. The CMS doesn't write good content for you, but it does clear away most of the clutter that distracts you from the task of writing good content.

C|net quickly realized that the tool they had built for themselves would be valuable to anyone who has large quantities of content that need to be published and kept up-to-date online, whether they are news sites, e-commerce sites (think Amazon.com), or corporate intranets. They licensed the technology to a company called Vignette, which is still the leading CMS provider today.

Up until recently, CMS's were not being used in online learning because the return on investment was not there. A CMS is usually an expensive piece of software both to buy and to set up. When all is said and done, it often costs anywhere from $150,000 to $500,000. (Some companies have easily spent a million dollars or more, depending on the purpose of the system and the elaborateness of the customization required.) In order to justify this cost, the system users have to be creating and maintaining a large enough volume of content that the system will pay for itself through the incremental savings of time and money required to produce each content item. In the last few years, significant numbers of organizations have started producing enough in-house custom e-learning content to justify the expense of a CMS. Between the increasing numbers of organizations that actually need a CMS for online learning and the fact that CMS prices have been coming down (albeit slowly), a market is beginning to grow.

Unfortunately, the needs of a CMS for Internet news publishing and the needs of a CMS for e-learning are not quite the same. Both the end product and the workflow process that produces it are unique to e-learning. Before we can benefit from c|net's innovations, we have to think through those differences carefully. We need to come up with a learning content management system, or LCMS.

What's Unique About Learning Content

One of the dangers of adopting an expensive solution that was created to solve a different problem is that, once the thing is installed, you may find yourself forced to adapt to what it can do rather than having it conform to what you need. So it's critical to identify the unique characteristics of learning content that will impact your needs for an LCMS.

Here are some unique aspects of e-learning content that we need to keep in mind:

Learning content is interactive. For the most part, c|net publishes words. There is very little to an article beyond those words and the graphical window dressing that goes around them. At most, a new publisher has to worry about inserting links to related articles. In contrast, an e-learning creator publishes experiences. Good training is interactive. This means that good training has programming logic to it: if the learner does x then the course will do y. Therefore, at the very least an LCMS should not make it harder for programmers to create interactive exercises, and in an ideal world it should actually automate some of the programming involved.

Learning content requires a broad range of presentation styles. You could probably list most of the types of articles (and therefore most of the types of templates) that need to appear in a typical news site. There are front-page stories, reviews, profiles, and so on. All in all, the number of templates you'll end up with is probably not very large. In contrast, e-learning must have a much broader range of presentation styles depending on the content, the learners, and the kind of learning that must take place. For example, a course developed to teach blue-collar workers who speak English as a second language about safety procedures in a factory would not look very much like a course developed to teach experienced stock brokers about the pros and cons of a new type of derivative. Nearly everything could change, from the use of audio and video to the navigation layout to the amount of content on each page. Consequently, the designer of a good LCMS must place a particularly high priority on making it easy for developers to create new presentation templates.

Learning content is particularly hard to write well. This statement is not meant to be a slight to journalists. I recognize that what they do is a highly skilled craft. However, journalists at least have a genre they can write to and widely accepted set of techniques they can learn. Most journalists can probably name a handful of great works of journalism without thinking too hard. In contrast, the majority instructional designers probably can't name a single example of e-learning that they would consider to be truly "great," and I doubt that any could name more than two or three. (I certainly can't.) We just don't have a Pulitzer prize for instructional design yet. As a result, a good LCMS must make room for experimentation and not force the designer to use certain tools or styles.

All of these differences add up to one important principle in LCMS design:

First, do no harm.

A good LCMS will not prevent course designers from using the same tools and techniques they have always used. In particular, it should not make it harder for web programmers to add custom code. Ease of use and reduced labor are important, but they are not terribly helpful in the end if you have to sacrifice the ability to create quality courses in order to get them.

Of course, this is a balancing act. Because a CMS is such an expensive product, there had better be some significant productivity gains to justify the cost. As we'll see, it is possible to get productivity gains without destroying quality. What you want is a tool that supports your existing methods and processes by clearing some of the clutter that distracts you from doing the hard work of the course design. In my experience, there are two areas where a well-designed LCMS can be enormously helpful. Taken together, these areas can reduce course development time and cost by 40-70%, depending on the specific circumstances, and make equally dramatic improvements on maintenance costs. This is because the traditional course design and production techniques are enormously inefficient and labor-intensive.

The Need

So what, exactly, are the problems of eLearning design and production that an LCMS can help to solve? Where's the opportunity to establish a return on investment?

Imagine the following nightmare scenario:

Suppose the Buysell Brokerage House has decided to develop a new custom trading computer program—Megatrade2000™—for all their brokers. It turns out that the program is quite complex to use. Buysell's Chief Knowledge Officer (CKO) decides that the such a high-visibility and mission-critical application should have a top-notch eLearning course bundled with it. He calls up his best training manager and tells her to have the course created and ready for launch in six months, when the new trading system is scheduled to go online. No problem; six months is plenty of time.

The first thing the training manager does is assemble her team. She needs a subject-matter expert (SME), an Instructional designer, a programmer, and a graphic designer. Unfortunately, since Megatrade2000™ is brand new, there aren't very many SMEs within the organization. The only one she can find is in the Singapore office, and he is busy supervising the late stages of the trading program's development, so he is very busy. He agrees to send whatever documentation his department has and to be available for occasional emails or conference calls, but that's about it. Likewise, programming talent is fairly scarce. The IT department wants an in-depth technical specification document before they are willing to even think about helping, and even then they probably won't have anyone available for at least 5 months. Since that would be cutting things too close, the wise training manager calls up her connection at Hire-a-Hacker and procures an HTML programmer for $100/hour. Fortunately, the situation is a bit brighter in the other areas. The department's best instructional designer just finished working on the new "Team Dynamics" course and is now available for the project. Likewise, there is a decent in-house graphic designer available.

The instructional designer starts by reviewing the documentation sent by the SME. Unfortunately, there are a lot of holes in it. And, despite having a beta version of Megatrade2000™ available to her, the instructional designer simply can't figure out how certain features of the program are supposed to work or, more importantly, why traders would find them useful. She makes a list of questions and calls the SME in Singapore that evening. He's not available. She emails him her questions. She doesn't hear back from him. She emails him again. He sends her brief, one-line answers, most of which aren't very helpful. Nevertheless, the instructional designer now at least has enough to go on to create a course outline, which she emails to the SME. He warns her that he won't be able to look at it for at least a week. Sure enough, a week later, he sends her a one-line email saying the outline looks "fine." The happy instructional designer then sends the SME-approved script for a review by the training manager who, in turn, passes it on to the CKO. The CKO, who has just returned from a seminar on publish-and-subscribe technologies, is very concerned that "publish-and-subscribe" appears nowhere on the outline. The SME rewrites the outline, adding lots of "publish-and-subscribe" references. She submits the outline again, which, after some more wordsmithing, is approved.

Six weeks. The instructional designer then writes scripts for the first module of the course. Unfortunately, there are a handful of spots where she's not sure how the program will work, so she inserts notes for the SME in the appropriate spots. She sends the script to be reviewed by the SME, who corrects her spelling in a few places but fails to respond to the notes in the script. When she calls him, he admits that he didn't see the notes. The script format is, after all, fairly complex and can be difficult to read. He spends an hour on the phone answering her questions. She rewrites the script and submits it to the training manager, who sends it on to the CKO. A week later, after not hearing anything, the designer hesitantly calls the CKO to check in. "Sorry," he says, "I didn't know the new script draft was ready." He reviews the script and complains that there still aren't enough references to "publish-and-subscribe." The instructional designer makes the requested edits.

Twelve weeks. The programmer and the graphic designer get module one up and running for review. They email the SME, the instructional designer, and the training manager to review the beta version.

All Hell breaks loose. The SME is furious. The module describes some critical functionality completely incorrectly and barely touches on some of the most critical functions of the new system. Besides, a large portion of MegaTrade2000™ has been completely redesigned in the last three months. "Isn't anybody paying attention over there?" he demands. The instructional designer, stung by the SME's criticism, is also angry because one of the sequences that the SME was angry about was not programmed the way it was written in the script. Somebody put in the wrong screen captures in the wrong order. The CKO, on the other hand, likes the program, but believes that publish-and-subscribe has become passé and should be removed from the course. "And please change the shape of those buttons. They're too drab."

There are now six weeks left until the deadline.

Clearly, there is a problem here that is worth solving.

Templating and Navigation Automation

At least part of the training manager's problem is the result of how long it takes to get content up on the web. A tremendous amount of time-consuming paper passing must take place before a single line of code is written. During all that time, many of the stakeholders have difficulty envisioning what their course will look like on the computer screen. Once the script is finally finished, it must be programmed by hand by an expensive resource. When the course is finally up, if there are changes to be made (which is common), then the expensive programming resource must be used for this as well.

At the very least, a good LCMS should enable you to get your content to the web faster. The first step in this direction is to automate some of the tedious, labor-intensive, and relatively mindless work that your skilled programmers have to do as part of the course development process. In other words, the tool should enable them to make a template that manages the global features of the course design—the look and feel and, more importantly, the navigation. The last thing you want is to have your programmers spend days manually inserting links into pages just so the "next" button will work properly.

A good LCMS should allow you to (simply, without programming) specify and edit the organization of your course. The vast majority of tutorials (which make up a sizeable chunk of most e-learning courses) have some mixture of a linear structure (sometimes derisively called a "page turner") and a simple non-linear hierarchical (or "tree") structure similar to most conventional web sites. Either way, it ought to be possible to specify the structure of the course in something that looks a bit like a conventional table of contents. Like a table of contents, you should be able to show the order of the content. Also like a table of contents, you should be able to specify different levels of content organization, with modules and lessons instead of sections and chapters.

Now imagine that your course interface could read this outline. In a "page turner," the "next" and "previous" arrows would always look to the outline to see the pages to which they should be linked. The page counter or progress bar would check the outline to see where the learner is in it. The menu would simply pull page titles out of the table of contents and display them appropriately. Even the headings at the top of each page could be drawn directly from the table of contents. Want to move a page? Just move it in the table of contents. Want to delete a page? Just delete it from the table of contents. All the navigation cues will automatically adjust themselves. This kind of functionality frees the course design team from a lot of time-consuming drudge work and enables them to focus on the hard parts, like writing good content and programming more the more elaborate and nonstandard parts of the course.

So far, so good. We've cut the programmers' time down very significantly. It would be even better, though, to make it possible for the instructional designer to be able to enter the content into those templates rather than having the programmer do it. Why not skip the scripting altogether and have web-based prototypes from the start? A common method for doing this in a CMS is to create a very simple and easy-to-use web-based entry form, where content authors can type directly into the template. The main advantage to this approach is that the people who create the course would not have to know how to use Dreamweaver or some other full-featured (and complex) HTML development tool.

This can be very powerful and worthwhile time saver if it doesn't force the course design team to give up the flexibility they get from using a skilled programmer and a professional-level development tools. We could imagine, for example, that the course designers might want to simulate part of the MegaTrade2000 application as part of the course. This would require some custom coding that would necessarily look and work differently than the rest of the course. A good LCMS should allow web designers to create their own pages using their own tools and put little snippets of code that can get the appropriate information from that table of contents when and how they need it. It's possible to design a templating system like this and still also enable non-programmers to create and edit content; the LCMS developers just have to work a little harder to make it happen.

Incidentally, you don't need to spend six figures for a fancy system in order to get this capability. For example, my colleagues at Christensen/Roberts solution created a similar solution using Javascript. Content authors can create and edit pages using the word processor-like features in Dreamweaver. They can edit the table of contents in a relatively straightforward text file. It's not as elegant as a full-blown LCMS but it also costs about two orders of magnitude less.

Whatever system you consider, whether simple or fancy, you should test it out by having both web designers and content authors try it out. For the content authors, have them add pages, remove them, edit them rearrange their order and their level in the hierarchy. For the programmers, have them create several different course page designs and add all the navigation features that should be able to use the table contents, using their authoring tools of choice.

Again, this functionality by itself can result in substantial savings of time and money. However, it is not, by itself a content management system. It is just an authoring tool. In order to have a full LCMS, you need to have collaboration and workflow tools.

Workflow Automation

Templating and navigation automation would have helped the Buysell team somewhat, but their bigger problem was communication and workflow. Content had to pass back and forth quite a bit for development, review, and editing. If the team could get the content on the web quickly and edit online, then the next logical step is to use the network to collaborate. The SME could use his limited time to review the content on-screen and provide timely feedback. Automated alerts could have told the CKO (and the rest of the team) whenever there was a new content draft to be reviewed.

There are at least three main elements to a decent workflow system—versions, tasks, and roles. Of these, the first is the most straightforward. You want to make sure that everybody is looking at the latest version of the content. With multiple task members working in parallel, it's easy to lose a change here or there, or to have some changes in one copy of the course and other changes in another copy. A decent LCMS makes sure that everybody is looking at the latest version with all the latest changes. It also stores the older versions. Sometimes you may decide that a recent update was a mistake and would like to be able to roll back to an earlier version. In other case, you may want to be able to have an audit trail that enables you to say exactly when the content was changed. (This can be particularly important with training on laws or regulations, where the organization has a legal responsibility to show that their employees were adequately trained according to the most current guidelines.) An LCMS should manage versioning for you. You'll want to think through just what your versioning needs might be. How important is that audit trail? How often will you be likely to roll the content back to a previous version? What kind of notes do you need your design team to enter regarding changes in each version? Write out some likely scenarios (which programmers call "use cases") and try them out with the prospective LCMS you are considering.

Likewise, the LCMS should handle all the tasks in the lifecycle of the course, including creating a draft, commenting on that draft, editing the draft, approving it for publication, and actually publishing it to a public site where the learners can get to it. These tasks will vary somewhat from organization to organization and situation to situation. Again, regulatory training is a good example. Some content may need to be approved by the legal department before it is published. You want your system not only to enable this step but to enforce it. It should not be possible to publish content that needs legal approval without first getting that approval. On the other hand, you don't want to force content that does not need that approval through the extra time-consuming (and expensive) step. So the LCMS should allow you to customize your tasks somewhat. Again, you'll want to write out some use cases and try them out.

Finally, you'll want to think about who should be allowed to do what. In your world, do you want to allow you SME's to edit the content themselves or would you rather have them enter comments and then leave it to your writers to control the words? Is the person who approves the content for release the same one who actually schedules it for publication? For each task, think through who should be allowed to perform it and who should not be allowed. The LCMS should enable you to create roles (e.g., content creator, editor, reviewer, and so on) and assign permissions to these roles.

You want to be careful here, though. Workflow customization is an area where software complexity (and cost) can spiral out of control pretty quickly. Keep an eye on your return-on-investment. Will the customization you are considering actually make your existing process more efficient, or is it just one of those features that you want to have because, now that you know it's possible, you figure you might need it someday?

The devil is always in the details. The two main functionality areas I have just covered—templating and workflow—are present in pretty much every CMS and LCMS. It's the particular implementation that makes all the difference. Make sure the features you're paying for will work in your real-world environment, and remember: First, do no harm.

Bad Ideas and Other Distractions

Now that we have some idea of what works, let's take a quick look at what doesn't. Given what we've covered so far, there are a few ideas that are popular in LCMS marketing that we can now filter out as unimportant noise. Some of these are good ideas in principle but put the cart before the horse, while others are just plain bad ideas. All of them lead to a sort of "instant coffee" mentality about e-learning design and production. The marketers want you to think that their tools can make it possible to whip up great courses effortlessly. The truth is that these tools can make it possible to whip up bad courses effortlessly. Designing good e-learning is intrinsically hard because it requires hard thought—something no software can do for you. Whenever a vendor offers you an easier way of doing things, you should look closely to see if they are doing so by genuinely cutting down on work that can and should be done by a computer or if they are just glossing over hard design decisions and reducing your instructional options.

Distraction #1: Importing from PowerPoint. This is a popular gimmick that sounds great as long as you don't think about it too much. If you are going to put out PowerPoint slides, why not just distribute the slides and be done with it? Why bother to import them into some sort of online learning system? Does the fact that you have imported them somehow imbue them with more educational value? I don't think so.

Now, it is true that basic, linear online tutorials, which are often an important part of a course, can sometimes look an awful lot like PowerPoint presentations. I suppose it would be beneficial if you could create those tutorials in the tool you're used to using and add the interactive pieces using more traditional web development software. Unfortunately, another typical side effect of importing as PowerPoint is that the output is very difficult for your programmers to modify should that become necessary. In general, PowerPoint is not designed for interactivity. It doesn't make creating good interactive online content easy. For the most part, it's more likely to get in your way than it is to help. There are other, better ways to make tutorial authoring easier without sacrificing flexibility.

Distraction #2: Repurposing for print. Another somewhat seductive idea making the rounds is that you can put your content into the system and effortlessly produce both print-based and online versions of the same course. It's certainly true that you can create technology that is equally happy generating a web page or a PDF. It's also true that some of the core content you will use for a web-based course can be re-used for a print-based course. However, don't be fooled into thinking that it's a simple as pushing a button. Anyone who has ever tried to repurpose print-based material for the web knows that you write differently for online delivery. Content that is written for use without modification both on the web and on paper is likely to read poorly in both media.

If you have a clear need for both print-based and online versions of your content, that's one thing. Buy the tools that can handle this, but also be prepared to still spend a significant amount of time tuning the content for each medium. On the other hand, if the idea of creating both online and print versions was one that was first suggested to you by a vendor, then think seriously about how much of a time investment you'll be willing to make in producing and maintaining both versions. If the answer is "not that much," then ignore this feature.

Distraction #3: Re-usable learning objects. Re-usability is a great idea in principle. Who wouldn't want to re-use their content? In practice, though, it's very difficult to achieve. It's hard to write content that can be mixed and matched with other content without sacrificing clarity and continuity. Additionally, the dirty little secret of re-usable content is that the SCORM standard that is supposed to make it possible really isn't up to handling re-usability well yet because it doesn't have any way for you to string these content bits together in a sequence. It won't be until SCORM version 1.3 comes out sometime in 2002—or possibly even until SCORM 2.0 comes out in 2003—that re-usability will start to become viable in a standard way.

I'll be writing more about how and when it makes sense to aim for re-usability in a future article. In the meantime, suffice it to say that unless you have a clear and well-developed plan for creating re-usable content, you probably shouldn't pay too much attention to features that supposedly promote re-usability.

All of this is not to say that an LCMS can't make building courses easier. It can. You just need to look in the right places to find efficiencies that make sense.

What I'd Like to See Next

So far, we've covered some core areas of functionality you can find in the LCMS market today. Some of these functions are useful; others are not. I'd like to conclude this piece by touching briefly on a few areas that very well could (and, in my opinion, should) be supported by an LCMS, but aren't yet (at least to my knowledge).

Wish list item #1: Bug tracking. "Learning Content Management System" is actually something of a misnomer. Unlike the content that is managed by a traditional CMS, learning content (or at least parts of it) also can contain significant programming logic, including simulations, tests, animations, and so on. Consequently, an LCMS should include some kind of workflow of identifying, tracking, fixing, and verifying the fixing of bugs in the code. (For a decent example of a bug tracking system, check out BugHost.com.) This shouldn't be too hard to add, since the LCMS has to have a workflow component to it anyway. It just hasn't occurred to the vendors to add this yet. I suppose this isn't surprising, given that some of them maintain the fantasy that we can author all of our courses in PowerPoint.

Wish list item #2: Interactivity support. Speaking of programming logic, it would be great to see an LCMS support the creation of more interactive content. Some vendors are starting to support relatively robust test, poll, and survey creation, and that's a start. However, what I'd really like to see is an authoring tool that supports the creation of branching simulations, decision support tools, and other interactive exercises. One interesting example of a client-side tool that could serve as something of a model is Quandary (http://www.halfbakedsoftware.com). This is a tool that is better experienced than described; I encourage you to take a good look at it.

Wish list item #3: A standard for sharing templates. Part of the problem with our industry is that everybody always has to teach themselves from scratch, reinventing the wheel in the process. We really don't have very effective ways of establishing best practices yet. Sharing courseware is a step in the right direction, but I'd love to be able to share course design templates (especially if these templates include the interactivity support I mention above). We'd essentially be sharing best practices in the form of code. Version 1.3 of the SCORM standard, due out some time this year, will include some enhancements that may make this sort of interactivity template sharing possible in a standardized way. We'll see.

There is no magic here, even in these future enhancements. To paraphrase Shakespeare, the fault lies not in our tools but in ourselves that we write mediocre eLearning. Good tools will not turn a difficult task into a trivial task. At best, it will common easy tasks into trivial tasks and difficult tasks into slightly less difficult tasks. That, however, may be enough for most of us.



Comments

  • There are no comments at this time.