Be Found:
Different kinds of search engines

How to design findable web sites and get ranked with the best

The basics: understanding search engines and spiders

Index or directory?

Setting the right keywords

Using robot protocols to prevent unwanted indexing

Use core HTML tags more efficiently

Optimizing tricks to avoid

Getting dynamic web pages indexed

The problem of unreadable content

The commercial options for SEO

Using htaccess for efficient redirection

The final word on SEO

Index or Directory?

There are two distinct kinds of search engine, and you need to know the difference between them so you can optimise your sites to take advantage of both.

The first is the text index variety, built using spider programs that read, sort and index the text content of every page they find. (Although to be precise other programs do the sorting and indexing work.) Google and Alta Vista are good examples of this approach; the indexes used by these sites are principally built by automated tools that read pages and work out ranking methods. The process of determining the ranking and relevance for different search terms is complex to say the least, but it is automatic.

The other is the directory variety, a categorised listing that sorts sites into different areas according to their content. Directories are maintained by humans rather than spiders and are organised according to large hierarchical directory structures. Two good examples of Web directories are the Open Directory Project at and Yahoo. These rely more on site submissions than the automated search engine sites, but, of course, content is found for these places using any means they deem suitable. Getting into a directory can take a very long time, as each item is checked and placed - at least in theory - by a real 'editor' rather than by a computer algorithm.

yahoogooglegrab1Yahoo showing Google:

The two forms of search engine do mix. Google and other index databases refer to directories for finding content, and directory searches will often give results from indexes (sometimes flagged as 'web results') as well as from their own site directory databases.

Looking for tips on searching more effectively instead? Read the Search Secrets pages.

Have you found the information on this site useful? If you like, you can make a small donation directly to my hosting bills! It would be deeply appreciated.