Tricks to avoid

Don't think you can come up with a way to fool a search engine spider. The odds are stacked against you and the risk is high.

As a page's text content is so important to getting high rankings in search results, some people try creative ways of hiding keywords in their pages to try to get more hooks without spoiling the way the page looks to the human viewer. As spiders just look at the raw HTML content rather than the rendered layout, it shouldn't matter that pale text containing half a ton of keywords is invisible on the white background - right? Unfortunately, no, that's entirely wrong. Less-than-reputable sites with irrelevant keywords have used this sort of trick for a long time. As a result, search engine spiders look for this sort of trick (and design concepts that look like this) and mark down or block sites that use it.

Don't create 'doorway' pages - that is, pages intended specifically to be indexed and to lead people into a site via numerous different entry-points. These include pages with auto-redirects, as these are seen as trying to fool an index into listing a site from a number of different places. This means you should avoid making initial home pages that wait a few seconds before taking the viewer to a different page. Even if you don't intend this as a trick, that's exactly how it is seen.

Many search engines use link popularity as a ranking method to calculate a page's relative worth. Link farms are pages devoted to hosting links to many other sites, specifically to make spiders think the referenced pages are more popular than they really are. On the face of it, link farms can seem like a good idea. However, this isn't the case, as this technique is regarded by search engine and directory owners as yet another form of URL spamming. The normal result, as you'd expect, is the downgrading of the linked URLs.

This doesn't mean you shouldn't have links pages with pointers to other sites - quite the opposite, in fact. Links are counted as important popularity indicators in many search engines. By all means arrange to share links between sites if you know other Web designers - this is a very effective way to improve your ranking with most worthwhile search engines and directories. However, be sure you include some form of useful content as well, such as snippets of information about the links. This shows the page is meant for human use rather than designed to trick spiders. Don't let it look like a link farm, with hundreds of URLs and no actual worthwhile page content.

You can use Google to see how many sites link to yours - or at least which sites Google knows do this. Just search for "link:" and your URL, as in link:www.vortex.co.uk. Apple's UK site (www.apple.com/uk) has around 2200 links to it in other sites that Google has found, which is not at all bad but doesn't begin to compare with the 89,000 for Apple's main US site.

When considering ways of getting your sites into indexes and directories, you should put aside ideas of tricks and subterfuge. Put your efforts into making the site more suitable in real terms by making it easy to navigate, easy to read, and full of useful and interesting content. That's the work that will serve you best in the long term.

Next: Commercial options

Introduction

(Plus: Reading Matter)

The Basics

(Plus: Index or directory?)

Setting the right Keywords

(Plus: Robot protocols)

Tricks to avoid

(Plus: Unreadable content)

Commercial options

(Plus: Using htaccess redirects)

The final word

site map

With sites that have content that causes problems for spiders - for example, lots of plug-in media, dynamic menus, database content and so on - it's worth making a site map containing links to all the key pages in the site. Keep the links descriptive and put a link to that page somewhere in your main page so spiders will find it easily. Call it something obvious, such as 'site map', and it will be useful for all kinds of visitors - human as well as spiders.

Unreadable content in web pages

Search engine spiders use the text content of a site as the basis for indexing. Graphic images of text can't be indexed, as they're nothing more than collections of pixels, whatever they look like to the human eye. If you use a graphic 'text' headline and want this to be accounted for, it's essential to make good use of the alt text attribute of the image tag. This data is available for indexing, and is used to varying degrees by most spiders. This means alt text is important for more than just user accessibility. The need for text also applies to sites with large amounts of plug-in content. As media shown using helper software such as the Flash plug-in can't be understood simply by looking at the HTML page, anything that relies on reading that code can't work with that form of media. The following examples show ways of using alt text for graphics and for plug-in media.

Graphic text

Graphic text: Dreamweaver and GoLive can set alt text automatically based on the imported file's name, and Freeway can do this for graphic text produced in the layout as well, but this isn't always ideal. If the imported file's name isn't appropriate, or if the graphic text in the Freeway layout uses fancy punctuation or quirky spacing for visual effect, you'll need to set the alt text data manually instead.

Embedded Flash

Flash movies are widespread, but their text content can't be read by the mainstream spiders. As such, you'll need to include text in some way in the pages that host the Flash files. There are ways to export text into HTML files, although these need work before they are spider-friendly. It's best to reserve Flash for where it's really necessary and use regular HTML as much as possible.

Alt text

Use appropriate keywords in the alt text. If you have an image of a camera on the page, don't just set 'camera' as the alt text; consider something longer and more descriptive. This will be more useful if images don't appear in a user's browser, and will provide more content for spiders to index. Be aware that alt text doesn't wrap when too long; Internet Explorer crops it and Safari doesn't show it at all.

Of course, alt text is used for far more than just broken graphics and search engines. It is also essential for people using non-visual browsers such as screen readers, so make sure your alt text is useful and informative - not just tricked out to impress the search engine robots! And finlly, Google uses alt text as part of the index data for its Google Images search engine.