[ formatted version ] [ home ] [ about ] [ forum ] [ RSS ] [ help ] [ search ]




[ Hear the podcast: Audio | Transcript ]

One of the most significant consequences of the widespread diffusion of digital technology is the vast amount of data generated as a result of ordinary things we do each day. One study has sought to quantify just how much data and came to a rough estimate of about 5 exabytes of new information produced annually and stored (mostly) on hard drives. Just how much data is that? If the seventeen million books in the Library of Congress were digitized it would yield about 136 terabytes of data. Using this as a benchmark, the annual information produced globally would be equivalent in size to the information contained in 37,000 new libraries the size of the U.S. Library of Congress book collections. Looked at another way, there is about 800 megabytes of recorded data produced per person annually. This is equivalent to a shelf of books 30 feet long. The amount of data produced each year is estimated to grow at a rate of about 30 percent.

Not long ago information was relatively scarce; the task of collecting, transmitting and storing data was costly and time-consuming. What data we did have at our disposal was cumbersome to evaluate. Although the value of carefully analyzed data was great, the opportunity to make good use of it in decision-making did not always avail itself. In contrast, today we exist in an environment that is data rich. The speed and declining cost of data collection, storage and transmission provide amble opportunities for managers to make decisions based on empirical data and analysis.

But soon we will be flooded with information--a digital tsunami. Individuals with access to the web already have millions of destinations, or domain names -- with more than 300 million host computers by January 2005 -- totaling more than eight billion publicly accessible web pages. In little more than a half-decade we have gone from information scarcity to an overabundance, and that fact alone has changed the way many of us go about our lives. Learning how to navigate this sea of information is paramount. As individuals, we need to understand how to find the information we seek with reasonable ease, speed, and accuracy. And businesses on the web must be confident that they can and will be found by potential customers who seek their goods and services.

The challenge of finding information has led to the early development of web directories and search engines. These tools have quickly become an indispensable feature of the Internet. The popularity of sites like Google and Yahoo! has given them a unique status because they serve users as a point of entry to the web. Knowing how they work is key to understanding how to search and how to be found on the web.

Finding a business on the web may entail no more than adding ".com" to its name. The problem of finding what we want (and being found) becomes trickier when searching for a product or service instead of a particular company. A simple search of a major web directory can yield hundreds of resultant links. That is usually more than we have the time or energy to investigate. Typically, search accuracy, in terms of the actual number of useful sites, can be very low. This is unsatisfactory for both consumers and businesses, alike. The key to being found on the web is knowing some basic elements about how to code your web site. These include, but are not limited to: picking the right "keywords" and positioning them correctly on your web pages, using meta-tags, including html links along with graphics, and submitting only key pages to search engines.

Sessions/Visits per Person per Month
Domains Visited per Person per Month
Web Pages per Person per Month
Page Views per Surfing Session
PC Time Spent per Month
Time Spent During Surfing Session
Duration of a Web Page Viewed
Active Digital Media Universe


Current Digital Media Universe Estimate


Source: Nielsen//NetRatings (January 2007)


Learning Objectives

  • Develop a basic understanding of how to search and how to be found on the Web;
  • Understand the strengths and weaknesses of different approaches to indexing the Web;
  • Identify techniques for improving the likelihood of a site being found by search engines;
  • Examine the common methods by which users search for information;
  • Learn about recent advances in search technologies.

Things to read:

The Google Legacy: Google Technology
Stephen E. Arnold | 08.31.2005

Search Engine Users
Deborah Fallows | 01.23.2005

Into the Mind of the Searcher
Gord Hotchkiss | 04.02.2004

Buying Your Way In: Search Engine Advertising Chart
Danny Sullivan | 05.30.2003

How Search Engines Work and How Search Engines Rank Web Pages
Danny Sullivan | 10.14.2002

How Much Information?
Peter Lyman and Hal R. Varian | 10.27.2003

Guest Lecture:

Yahoo: Past, present and future
Jerry Yang and Terry Semel

Case study:


Hungry minds:

The State of Search Engine Marketing 2004

Paid Placement in Information Gatekeepers
Hemant K. Bhargava and Juan Feng

Placing Search in Context: The Concept Revisited
Lev Finkelstein, et al.

Search Systems in Information Architecture for the World Wide Web
Louis Rosenfeld and Peter Morville

The Anatomy of a Large-Scale Hypertextual Web Search Engine
Sergey Brin and Lawrence Page

Searching the World Wide Web
Steve Lawrence and C. Lee Giles

Accessibility of Information on the Web
Steve Lawrence and C. Lee Giles

Information on the Fast Track
Bruce Schechter

Tips for Searching the Web
Steve Lawrence and C. Lee Giles

Look it up:

Deep web

Domain Name System

Search Engine Optimization

Semantic Web

Web crawler

Places to visit:

Search Engine Watch

Previous topic:


Next topic:

Digital Design

Course information:

© 2007 Michael Rappa
Page last updated: