How to resolve Restricted by robots.txt

Discussion in 'Internet Marketing' started by asad, Feb 29, 2012.

  1. asad

    asad
    uix_expand uix_collapse
    Member

    Joined:
    Jan 22, 2012
    Messages:
    199
    Likes Received:
    8
    Some of my website outbound links are nofollow. it was ok from any error. But after two months i got some of those urls listed in "Restricted by robots.txt" in webmaster tools. Is these errors effect any SEO. And what is the way to resolve it.
     
  2. vishal23

    vishal23
    uix_expand uix_collapse
    Member

    Joined:
    Nov 4, 2011
    Messages:
    96
    Likes Received:
    3
    I don't think so but if they are nofollow then why you use it because Google can get it usefulness.
     
  3. ThirdSEO

    ThirdSEO
    uix_expand uix_collapse
    Member

    Joined:
    Feb 10, 2012
    Messages:
    239
    Likes Received:
    52
    Your robots.txt file is probably blocking crawlers from accessing your pages. To allow complete access to your site, make sure your robots.txt file says this:

    Code:
    User-agent: *
    Disallow:
    
     
  4. AliceNolan

    AliceNolan
    uix_expand uix_collapse
    Member

    Joined:
    Oct 7, 2011
    Messages:
    53
    Likes Received:
    1
    I think that this should be done through source code. But if you are sure that your pages are blocked by robots.txt, then you have to amend robots.txt.
     
  5. John Dale

    John Dale
    uix_expand uix_collapse
    New Member

    Joined:
    Feb 3, 2011
    Messages:
    18
    Likes Received:
    0
    Re:

    The contents of the robots.txt file look correct to me and everything seems to be indexed fine.You can also do like:

    User-agent: *

    Disallow:

    Sitemap: your website name with http/sitemap.xml
     

Share This Page