How Deep Do Search Engines Spiders Go?

[PowWowPalace]

I have a forum and was wondering how many forums - sub forums deep (or categories) do search engine spiders go? Ecspecially Google, Yahoo and Bing?

Example:

Forum Home>Travel>United States>Nevada>Las Vegas>Hotels

Will the spiders go to Hotels to query whats there?

Will the spiders only query up to Travel?

Any input would be great, thanks.

[ZakShow]

I think it's going to reach Hotels section, you just need to generate and submit a sitemap.

[pendelton]

The search engines will spider all the pages on your site, not always the first time, but they will.

To increase the speed a site map is a must, as well as back links to the deeper pages on your site. Don't always use the index as the page that you link too for back links, use the other pages that fit with the area that you a linking from.

[PowWowPalace]

Thanks for all the input... very useful info.

[DEADMAN]

I have a forum and was wondering how many forums - sub forums deep (or categories) do search engine spiders go? Ecspecially Google, Yahoo and Bing?

Example:

Forum Home>Travel>United States>Nevada>Las Vegas>Hotels

Will the spiders go to Hotels to query whats there?

Will the spiders only query up to Travel?

Any input would be great, thanks.

As far as I know, they will go through every pages of the site and with keywords and meta-tags.

[hcjunkie]

this info is good to know. Thanks.

[axeline]

THE SPIDERS:

All the spiders "deep crawl", which means that they crawl several layers into your website. The exception to this from the above listed search engines is the spider of the search engine, Go.

Only Alta Vista "Instant Indexes", which means that the information is put into the database of websites within days. All other search engines take longer.

Excite Inktomi, Go and Lycos cannot crawl nor index frame pages. Alta Vista, FAST, Northern Light can. It is possible to circumvent the problems with framed pages of some search engines by using the <NOFRAMES> tag.

If you have an image map as one of your main pages for links to other areas of your website, you will have problems indexing your pages with the spiders of Excite, FAST, Google, Inktomi and Lycos. They cannot follow the links in an image map. However, links in an image map can be read and followed by spiders of the following search engines: AltaVista, Go and Northern Light.


_____________________
baby dress up (http://dolls.dressup.me/) | baby dress up games (http://dolls.dressup.me/) | baby games (http://dolls.dressup.me/) | dress up dolls (http://dolls.dressup.me/)

[opnil]

i also agree with that the spider of the google deeply visit the every page of the site.

[simplyjo]

The depth of the crawl actually depends on what you specify in the robots.txt file. Exclusions mentioned here will not be crawled.