Separate names with a comma.
Discussion in 'Search Engine Optimization' started by rkdolla, Dec 20, 2013.
Please share your answers..
You really don't want to stop search engines entirely. I think you want to keep them out of some pages. You use robots.txt file to do that.
Also you can use Sitemap.xml to tell them what you do want indexed.
you can also use the .htaccess file to block undesired web crawlers.
You don't know his needs and whether this is necessary for his implementation or not.
You can put rel="No follow, no index" attribute in meta tag.
You can use robots.txt file
or you can do it through search engine webmaster tools.
Either You Can Put put rel="No follow, no index" In Meta Or You Can Put .htaccess File to Block Crawler or Even you Can Put Robot.txt File In That.
The Pages which you do not want search engine to crawl then you need to use ROBOT.TXT file.
As per I know we can't do stop crawling through webmaster tools. If there is anyway in the Google webmasters tool then its in URL Parameter section which is useable only for dynamic urls.
If you don't want to crawl your website from search engines, you can put your site pages in the robot. text file.
if you want that search engine not crawled your webpages so you can use robot.txt.
best and fastest way is to put <a href="..." rel="nofollow, noindex"> tag.. b coz of that crowler won't crowl ur page
This thread is now closed. If you have anything new to add, let me know and I'll re-open it.