Search Tools

Overview

What do I need to know about search tools?

Purpose	Components	Comparisons
Methodology	Types	Reading resources

Purpose
Designed to retrieve web sites in response to user-generated requests. Each search tool makes available to users a database or collection of web sites. The collection of web sites available varies and the largest indexes only 30% of the 1 billion web pages. In today's digital culture, search tools are big business because they promise to connect users with information. Search tools also realize that their longevity is dependent on user satisfaction. Search tool companies concentrate money and time into research tracking what their users are looking for what and how they attempt to find the answers to their questions. As we look at search tool evolution we see most trends are in direct response to searcher behavior and expressed needs. Search tools are promoted through advertisements in magazines, newspapers, television, and web sites.

< top of page >

Methodology
Search tools accomplish their purpose in two ways: automated collection and human submission. In automated collection, the search tool uses computer programs called spiders or crawlers or robots to act as collecting agents and automatically gather information on the Internet. The spiders and robots search through the html code of web sites seeking significant terms. The results are returned to the home indexing site and organized according to ranking algorithms .

One of the most common misconceptions among searchers is thinking that the search tool ventures out to the live World Wide Web to retrieve sites about their topic. Quite the contrary, upon receiving a user's search terms the search tool searches its own internal database of assembled web sites and not the live World Wide Web. As a result, it becomes increasingly important to determine how frequently a search tool refreshes its database.

Ranking factors might include word order, titles, headers, meta tags, link popularity, and frequency of update. Ranking algorithms are considered highly proprietary and are not usually available to the public. In fact, users may note that a result in in one search tool is given a higher ranking, usually in percentage form, than the same result appearing in a different search tools result list.

Search tools also invite individuals to submit sites for inclusion within a search tool's database.

Search tools can be stopped by inserting robots.txt code that prevents spidering software from accessing the site.

Results Ranking in Web Search Engines
Martin P. Courtois, Michael W. Berry Online May 1999.

< top of page >

Components
Search tools are comprised of four primary parts:

collecting agents	Software programs called spiders, robots, or crawlers, which search the HTML code of web sites.
indexing unit	Organizes and includes the data collected in the search tool database
matchmaker	Receives the user's request, searches the search tool's index of web sites and responds with sites that match a predetermined algorithm.
home page	the user's first encounter with a search tool . It is on the home page that the user reads the on-screen directions and enters a search term or browses from the subject guides available

For information contact jegan@nvcc.edu
Last revision April 01, 2000