by Darlene Canning
- Trends in the development of search engines — past, present
- Future enhancements to search engines, with particular
attention to the semantic web and developing knowledge networks.
As the Web grows and the information world changes, search
engines must continue to evolve and adapt to change. Although
they have undergone considerable improvements, complaints and
frustrations persist in searching the Web. Part of the problem
arises from the fact that commercial search engines are capable
of indexing only a small portion of the Web at any one time. The
sheer volume of information limits what each search engine can
retrieve. This unprecedented growth of information is only
expected to continue. Web search engines must evolve and become
even more intelligent to overcome the problem of size and many
other challenges that are encountered in locating information on
The popularity of web search engines waxes and wanes based on
the value of the information they retrieve. New search engines
continue to appear and existing search engines constantly
improve their searching capabilities. Consequently, new and
greatly enhanced search engines with better functionality may
surface at any time to compete with, or replace, the search
engines currently in vogue. As the almost insatiable quest for
information continues, new types of search engines are
continually being developed.
So far, we have seen the emergence of what are called "second
generation" search engines. In other words, the level of
sophistication of these search engines sets them apart from
earlier versions. Thus, we have seen a search engine like
AltaVista lose its "best search engine ranking" to Google when
Google developed an indexing tool with a better relevancy
ranking. AltaVista only became widely available in 1995, and yet
is credited with creating a revolution in web searching. It was
both big and fast; furthermore, searches could be written in a
natural language as well as a Boolean search statement. User
satisfaction with web searching was greatly enhanced when
AltaVista appeared on the scene.
The following table provides a rough chronology of
significant events in search engine evolution to this point in
Important Dates in the Development of Web Searching
Tim Berners-Lee launches the
World Wide Web
Archie created by Alan
Emtage at McGill University. Archie provided access
to files that were stored on FTP servers. At that
time, the primary method of storing and retrieving
files was via the File Transfer Protocol (FTP). This
is still an accepted way for computers to exchange
files over the Internet. Archie scoured FTP sites
across the Internet and indexed the FTP files it
Veronica and Jughead
provided indexes to Gopher sites.
Lynx, the first text
Mosaic, the first graphical
browser, becomes available. Marc Andreessen and his
team at the National Center developed mosaic for
This marks a major turning point in
the development of the World Wide Web. Many events
occurred that year which moved the Web to the public
O'Reilly launches "Internet
in a Box" which was designed to assist users to have
access to the Web at home. That same year, many
Internet Services Providers (ISP's) appeared on the
Ed Kroll publishes his book
Whole Earth Internet Catalog. This book
brought the internet to the general public by
describing the use of FTP, telnet and many other
tools that would bring the internet and the world
wide web to the lay user.
By December, Marc Andreessen
has co-founded Netscape Communications and
Netscape Navigator 1.0 has been released as a
free resource on the Web.
Researchers at Carnegie Melon
University launch Lycos.
Yahoo, Galaxy, Yahoo and
Lycos all appear as search tools.
Compaq computers are bundled
with Netscape Navigator when sold.
Windows 95 computers are
sold with Internet Explorer on the machines.
Alta Vista arrives and
offers many advanced web search features
Excite and InfoSeek
become available for the general public.
Google appears on the Web, a
research project at Stanford University.
Ask Jeeves is launched as a
"natural language search engine.
Northern Light is another
new search engine with many improved features.
Yahoo's own webcrawler is
launched. Yahoo recently acquired the Inktomi
webcrawler, AlltheWeb, AltaVista as well as
Overture; this integration results in a greatly
diminished number of major competitors in the search