Book Review: 'Click: What Millions of People Are Doing Online and Why it Matters' by Bill Tancer

To paraphrase Nietzsche, when you look into the World Wide Web, the Web also looks into you.

Every day millions of Internet users click on millions of web pages. This results in more than people being connected with websites. It creates a tremendous aggregation of data about what people are thinking about.

In 2010 almost every organization, both commercial and non-profit, has a web site that affects the organization's ability to achieve its aims. As such, information about the habits and preferences of web users—and the ability to interpret that information—is highly valued.

Web Mining
Web mining, "the application of data mining techniques to discover patterns from the Web" (Wikipedia), is one of the more important new fields that have emerged in the past few years. One of its leading practitioners is Bill Tancer, general manager of global research at Experian Hitwise, a leading global information management company that provides clients with "competitive intelligence" necessary to survive and thrive in a wired world.

Tancer's book Click: What Millions of People Are Doing Online and Why it Matters is intended to be an introduction to Web mining (in the past Tancer has contributed columns on "the science of search" to Time.com). He writes in a straightforward, comfortable style that readily conveys his tremendous enthusiasm for the collection and analysis of Web traffic data.

"As the internet moves from a vast group of static pages we did little but read, to an environment where users are posting volumes of data about their personal lives, I have an ever growing, rich database from which to understand society, or more specifically, what people are thinking about at any given moment," he writes.

One example Tancer uses to illustrate the power of Web mining to contradict conventional wisdom deals with the search for prom dresses. Tancer assumed that web searches regarding prom dresses would begin in March and April, the months before most proms are held. Yet one of his routine data-mining efforts found a surge in prom-related inquiries on the Web in January. He arrives at a number of explanations, one of which is the Web itself.

"Prior to the Internet, the prom dress shopping seasons was dictated by the retailer. Upcoming prom fashions would begin showing up in stores in late March at the earliest. With the advent of the Internet, prom fashion merchandise and information is available 24x7x365."

A chance meeting with the general manager of the online teen division for a publisher provides another piece of the puzzle. Since prom advertising is among the most lucrative for such publishers, teen magazines had begun printing prom advertising in late December.

Using the Data We Have
Our society is data-driven, but as Tancer notes, traditional polling practices are in decline. In the United States, he points out, this is partly a consequence of the Do-Not-Call Implementation Act of 2003, which allows people to have their names placed on lists that prohibit telemarketers from contacting them.

Another cause for the decline in traditional polling is the phenomenon of "cord cutters"--people who use only cell phones and do not have landline phones. Since many random-calling surveys only contact landline phones, this significantly undercuts pollsters' ability to obtain quality data.

All these changes make Web-based data studies all the more important.

Web mining and its relationship with education, as a topic, has received less attention than it deserves, probably because it does not fit into an existing paradigm for many educators—even those who consider themselves savvy about the Web in general and e-learning in particular. How many of us who teach using learning management systems like Blackboard ever bother to use the Web mining tools available in the LMS to better understand our students?

However, this may be changing. A cursory search on the internet using "Web mining" and "education" brings up links that indicate educators are beginning to recognize the value of Web mining. For example, the journal The Internet and Higher Education is planning a special issue on Web mining and higher education.

Tancer's book is a helpful piece that should be read by the e-learning community. It's a good starting point for a conversation that is long overdue.