|
1. Why
do I need a search engine?
For the same reason you need a
card catalogue in a library. There is lots of great and useful information in a
library, but it's physically impossible to examine all the books personally.
Not even the most skilled web-surfer could hyperlink to all the documents in the
aptly named World Wide Web. There are billions of pages on the Web. And
every minute of the day, folks are posting more.
The search engines and
directories help you sift through it all to find the specific information you
need.
2. If it's
impossible to examine all the documents on the Web, how do the search engines do
it?
They use software programs known
as robots, spiders or crawlers. A robot is a piece of software that
automatically follows hyperlinks from one document to the next around the Web.
When a robot discovers a new site, it sends information back to its main site
to be indexed. Because Web documents are one of the least static forms of
publishing (i.e., they change a lot), robots also update previously catalogued
sites. How quickly and comprehensively they carry out these tasks varies from
one search engine to the next.
3. Which
search engine is "the biggest"?
They'd all like you to believe
they're either "the biggest" or, if they can't possibly claim that..."the best."
Google currently claims to
index over 3.3 billion pages, while Alltheweb.com (one of Yahoo’s many search
properties) claims to index over 3.1 billion. Search engines which demand that
site owners pay to have their sites indexed, such as
AskJeeves, index fewer pages.
4. How Do Search
Engines Work?
There are
many detailed explanations of search engine technology, including keywords,
clustering, relevancy ranking, meta tags and more. To answer that question we
recommend going to sites like
searchenginewatch to get the detailed answers.
5. What's
the difference between a Web directory like Yahoo and a Web search engine like
Google?
There is less difference now than
there used to be, because many search engines, including Google, have built
large subject catalogues to help you search. But think of a Web directory as a
subject catalogue--something like the subject catalogue in your local library.
Yahoo started out as a directory but is now de-emphasizing that aspect of their
broadly-based business.
However, directories such as The Open Directory aka dmoz and the Google
Directory attempt to organize Web by dividing it into topics and subtopics.
Some examples include: Arts, Science, Health, Business, News, Entertainment. If
you're looking for information on the Web that fits neatly into an obvious
subject or category, go first to a web directory.
Think of a Web search engine as an index that enables you to seek out specific
words and phrases. With the search engine's help, you can locate individual
appearances of such words in documents all over the Web.
This can be both a blessing and a curse--but it's more commonly the latter! You
are likely to get far too many hits. Or you might discover that your keyword
has meanings you didn't anticipate. Rarely, you might get no hits at all.
In brief, here's a quick run-down of some well-known general topic search
engines:
Yahoo's directory, the Google Directory, and the Open Directory Project (dmoz)
are web directories -- essentially subject indices. They began as attempts to
catalogue important/useful pages on the Web. Search on a subject or topic. If
you know exactly what subject you're searching for, and have a good sense of how
to find your subject within a hierarchies of larger subjects, a directory is a
good place to start.
Google was one of the newer search engines, but it rapidly become the favorite.
In fact, the word is commonly used as a verb, a synonym for searching: "I'm
going to google the web to find the info I need."
Google is thorough and fast. Its technology considers pages that are linked
from many other sites to be more important than pages that only have a few links
from other sites. In other words, if many webmasters consider a website
valuable enough to create a link to it, Google considers that a good reason to
justsify a high ranking for that site.
Confusingly, Yahoo’s search functionality was for a couple of years based on the
Google Search Engine! In fact, Yahoo was one of Google's investors. As of
March, 2004, however, that partnership ended, and Yahoo is now developing their
own search technology, which will probably be based on search algorithms they
obtained when they acquired Inktomi.
Meanwhile, Microsoft’s search and MSN.com have also been based on Inktomi's
technology (Inktomi provided the technology for one of the hot search engines of
the 1990s, Hotbot. Microsoft reports that it is working to develop its own
inhouse search technology to replace Inktomi, but there is no fixed date for the
cutover.
Alltheweb is also one of the newer search engines. It claims to be faster and
more efficient than other search engines, with the largest index and the most
rapid look-up times. Alltheweb has been acquired by Yahoo, and may at some
point be integrated with Inktomi.
AltaVista was the favorite of web searchers a few years ago, but Google quickly
became the favorite for web surfers. It still provides users with excellent
search refinement capabilities, though. Altavista has also been acquired by
Yahoo.
Lycos was one of the original Web search engines, but has morphed into a general
web portal with something of a European focus (it was bought by a Spanish
company now called Terra/Lycos). Its actual search results are based on the
Inktomi engine.
There are also the concept-based search engine (excite.com) and the special
interest (trekSearch.com) search engines. These were an interesting idea in
their time, but they are now web history.
6. How Can I
Find Out More Details About These Search Engines?
Sites like
searchenginewatch,
and
searchengineguide review technology, events and
tips regarding search engines and we recommend starting with them.
7. How can I
get a higher search engine ranking for my website?
It used to be true
that if you posted a page on the Web, sooner or later the search engines would
find it and index it, as long as it's not too deep into the particular site's
hierarchy (i.e, "deep" refers to how many clicks your site is from the site's
main page).
To some extent,
this may still be true. But as the Web has exploded in size, it's obvious that
less and less of it is actually being found and indexed by search engines. Now
a website developer has to be more proactive than ever before about getting
listed by search engines and directories. In many cases, this means
(unfortunately) that you have to pay a fee to get listed.
Most of the
popular Web search engines still provide a way for you to add your own URL.
Sometimes this is obvious on their initial page; other times you have to drill
down into the site to find out where and how to do this.
We continue
to recommend that you use meta tags in the "head" of your document. With meta
tags you can provide your own keywords for the search engine to index you with.
More importantly, you can also provide a description of your site that the
search engines can use to show to Web surfers or researchers when your site
comes up as a hit.
However, meta tagging does
not provide you with any guarantee that your site will be listed or highly
ranked. There has been so much abuse of the keyword meta tag in recent years
that meta tagging is not as helpful to web developers as it used to be in the
late 1990s. Not all search engines index the meta keyword information (Google,
for example, does not).
You can hire
various amateur and professional "search engine optimizaton" professionals who
charge a fee in order to get a higher ranking for you. Some of these fees are
quite hefty, and there is rarely any guarantee. Beware, also, that you don't
hire someone who uses unscrupulous tactics to try to
"spam the search engines." This could result in
your site getting banned from search engine indices.
The best way to get people
to find your site is to provide useful, well-organized, well-written,
well-designed information. The more folks who find your site and link to it, or
send around the URL to their friends, the more visitors you'll have. Word of
mouth (actually, word of email/blog/forum) is very powerful on the Web! |
