ACM Logo  An ACM Publication  |  CONTRIBUTE  |  FOLLOW    

The Awards Dilemma

By Bob Little / January 2009

TYPE: OPINION
Print Email
Comments (6) Instapaper
Winning awards is great fun, not to mention a welcomed boost to one's professional (and personal) ego. November saw the U.K.'s corporate learning world in the grip of awards season: chief among them were the E-Learning Awards (held in London on Nov. 6), the WOLCE Awards (in Birmingham on Nov. 19), and the Personnel Today Awards (in London on Nov. 27). There are some other significant events scheduled in the United States.

So, what of the awards process?
Are these awards fair and objective?
Are they influence the professional growth of the field and the recipient?
Does the cream truly rise to the top through the awards process?

As a general rule, we should believe that the decision-making process for every award is fair in the sense that judges try to be objective and treat each submission on its own merit. However, from personal experience as a judge in some business awards (not in the e-learning world) a few years ago, I witnessed some of the judges expressing antipathy toward at least one of the entrants—so, regardless of the worth of that entry, it was never going to win.

I have no doubt that the judges of some of these U.K. awards do make a real effort to be fair. But it was interesting to learn that, in one of the recent U.K.-based ceremonies, a judge allegedly wrote the entry for one of the entrants and, remarkably, that entry won!

For awards to have a respected presence within their community, they must be significant—at least to the entrants. Otherwise, no one would spend the money to enter the awards and buy seats at the awards event, thus making money for the awards' organizers. Unfortunately, buyers of e-learning-related products tend not to come from the e-learning world, so they are only influenced by the publicity surrounding the award winners, not by the awards themselves.

Presumably, the cream of the e-learning industry in the U.K. tends to receive the majority of the awards. But it's a very small world, with judges rubbing shoulders on a daily basis with major providers. As a result there are relatively few 'surprises' when the awards are ultimately announced.



Comments

  • Fri, 30 Apr 2010
    Post by Irfan

    dear sir i dont know exactly how appropriate it is to ask you about ht e comment title. T explain, i need to know what should be the rules or measures that may decide giving a particular award to a person. like for example, if a person is to be awarded in the category of "GALLANTRY" and the awards are say fro example Gold, silver and Bronze then how will we decide the individual is to be awarded which medal. like wise other fields e.g art, music, public service, scientist etc etc. hope i have clarified my point. looking forward to hear from you Irfan

  • Tue, 10 Feb 2009
    Post by Lisa Neal Gualtieri

    I just read about an e-learning contest where the assessment process was specified. It is the 2009 SIGTel Online Learning Award Competition. Criteria for Assessment: Entries will be evaluated using a rubric based on the following attributes (weightings are shown as a percentage of final score): * Detailed description of the online learning activity from initial planning to the final evaluation (40%) * Description of how the online aspect of the learning activity enhanced the regular classroom program (15%) * Description of how the online learning activity was evaluated and the extent to which original aims and objectives were met (20%) * Consideration of what the entrant learned from the online learning activity in their classroom with comments and recommendations for development and involvement in future similar activities (25%) Descriptions should be sufficiently detailed to allow another educator to follow the same process, modifying it to meet the requirements of their own classrooms.

  • Fri, 30 Jan 2009
    Post by Bob Little

    Bob Little replies: I agree wholeheartedly with Judy Unrein''s sentiments. In an ideal world there would be guidelines regarding conflicts of interest – and by which everyone would abide. Moreover, any ''less than scrupulously honest'' behaviour would be known about – and publicised widely. I''m delighted that, to Judy at least, ''awards matter''. I sincerely applaud that view because I like to believe in a ''Disneyesque'' world where, despite adversity and sharp practice, virtue is always rewarded. However, when I was starting out as a freelance writer and PR consultant some 20 years ago, I asked my mentor – a man who had had some success as a film publicist in Hollywood before returning to the UK to set up his own PR agency – if he thought I should become a member of the UK''s institute for PR practitioners. His reply was cynical but experience has proved it true. He said: "If you''re a good PR consultant, you''ll get work. If you''re not, not amount of qualifications will help you." For ''qualifications'' read ''awards'' and this story applies to this issue too. As it happens, I have been a member of the UK''s Chartered Institute of PR for many years – but no potential client has ever asked me about my professional qualifications and I have never been awarded a job because of the awards that I have won. Nonetheless, as I have said, awards not only give a useful boost to producers'' egos and provide an excellent excuse for a party (the value of which, in these days of economic downturn, cannot be underestimated) but they also provide a valuable opportunity for building brand and producer recognition in the marketplace. The issue of awards'' judging criteria is highly complex. For one thing, your views depend on whether you believe that the awards are a ''level playing field'' to define excellence or have been staged to get the greatest support from the largest, wealthiest producers. For example, any fee to enter awards is usually nominal. The awards organisers make their money from the tables sold at the awards'' dinner. So they prefer to have an awards shortlist made up of companies that will take one or maybe two tables of ten people at the dinner. This always puts the small, innovative, creative but impecunious producer at a disadvantage from the start. Moreover, however ''objective'' the judges in their decision, I have known it done for the organiser to influence or even overturn this to appease the demands of capitalism - which, it must be said, we all endorse to some degree of other. I also agree with Judy''s comments on the judging criteria. Within the e-learning and related corporate development worlds, it is difficult to decide whether a project involving a large number of learners is more worthy of an award than one which uses state-of-the-art delivery technology or, indeed, whether success should be measured in numbers of learners ''processed'' or the impact of a learning initiative on an organisation''s sales and/or profitability. I would prefer not to suggest which awards I feel are the ''best structured''. I have little knowledge of the U.S. award scene but, within the UK, I can say that the judges of the E-Learning Awards are all chosen from the membership of the eLearning Network, the UK''s professional association for users and developers of e-learning. I know that they all take their job seriously and I believe that their judgement is worth having. However, the whole ''awards scene'' is a lot more complex and, perhaps, Machiavellian than the Disneyesque world view will allow. Nonetheless, speaking as a realist, faith is an important cornerstone upon which to build professional life and, for positive and not so positive reasons, awards are not going to go away. Bob Little

  • Thu, 29 Jan 2009
    Post by Tom Werner

    I direct the Brandon Hall Excellence in Learning Awards, and I certainly appreciate the concerns about whether any given awards program is operated fairly. I would say that there are a number of variables that affect the operation of any awards program: * The appropriateness and specificity of the judging criteria. (This seems obvious, but it''s challenging. For example, is this a good judging criterion for an e-learning course: "Objectives. How clearly stated are the learning objectives?") * What the entrants submit. (The bigger the submission item, the bigger the judging task to judge it.) * The available time of the judges. (Most volunteer judges can donate roughly 8 hours to judging.) * The number of judges needed. (The bigger the judging task, the fewer entries a judge can judge and thus the more judges needed. The more judges needed, the harder the task of supervising them.) * The qualifications of the judges. (This includes knowledge and experience, as well as lack of conflicts-of-interest.) Each awards program has to manage these elements. And inevitably the elements affect each other. For example, less-clear judging criteria mean relying more on the judges'' judgment. The more judges needed, the less attention can be given to an individual judge. And so forth. I think anyone can determine the fairness of an awards program by asking its director to describe its process. Then you can judge based on the clarity and transparency of the answer. I would say though that awards are always subjective in that they rely on the judgment of judges. So ''fair and subjective'' is probably a better expectation than ''fair and objective.''

  • Fri, 23 Jan 2009
    Post by Allison Rossett

    I''ve seen many awards judged and think that bias and cheating are not the problems. I see the issues, instead, in how labor intensive the application process is and how hard it is to see inside the reports. Often, I think, winners are people who had staff to prep these applications. Independent consultants or very busy organizations probably will NOT be assigning people to construction of these packages. As a judge, I''ve struggled to see quality and truth from afar. I won''t belabor it, but just leave it at admitting how hard it is and how unsure I have been. Should we get rid of awards? Nope. But I wish we could conduct the process differently. Maybe briefings with an opportunity for Q & A?

  • Thu, 22 Jan 2009
    Post by Lisa Neal Gualtieri

    People make decisions about which moves to see based on awards. But they also make vendor decisions. I was particularly interested in your opinion piece because I received an email from someone who has made an LMS decision based, in part, on an award that he thought had been bestowed by eLearn Magazine. (It was another magazine with a similar name, as we soon clarified.) But it made me realize how much weight these awards can have, a concern when the process is not transparent.