When a search engine crawls the Web for sites, it uses a spider. Of course, this is not a fuzzy arachnid, but an application that searches through the Web, cataloging the content on pages and the links between them. However, the important keywords and links displayed on your web page may not be visible to search engine spiders. That is why it is essential that you run a spider simulator against your site. This tool allows you to see your website through a spider’s eyes and get an idea of how the search engines see your site.
From the text to the links, all of the “spiderable” content will be listed right before your eyes. Hopefully, you will have a very extensive list. If not, you have some work to do.
In addition to the quantity of text and links, the spider simulator will show you if Google, Yahoo! and the others are seeing the right content in the right order. This is again part of marketing to search engines, and determines your ranking.
If this sounds familiar, you may be reminded of the XML sitemap discussion. While the sitemap tells the spiders what pages are on your site, this type of search tells the spiders what content is on those pages. Very useful information.
We have provided links to free spider simulators below to get you started. Just type in your URL, and go.
Learn More
www.webconfs.com/search-engine-spider-simulator.php
www.seochat.com/seo-tools/spider-simulator
