How to block subdomain in Robots.txt

Discussion in 'Internet Marketing' started by Andym, Nov 2, 2011.

  1. Andym

    Andym
    uix_expand uix_collapse
    Member

    Joined:
    Jul 22, 2010
    Messages:
    220
    Likes Received:
    11
    User-agent: *
    Disallow: /

    =================

    This is how my current robots.txt file looks like. I want to block a subdomain (i.e http://subdomain.mywebsite.com) using robots.txt, I don't want even a single page of my subdomain to be indexed in Google.

    I added the above text to prevent my Subdomain from indexing... but it still got indexed into Google. What should I do to entirely prevent my Subdomain from Indexing.
     
  2. BODHost

    BODHost
    uix_expand uix_collapse
    Member

    Joined:
    Oct 18, 2011
    Messages:
    186
    Likes Received:
    7
  3. cesspadilla

    cesspadilla
    uix_expand uix_collapse
    Active Member

    Joined:
    Sep 8, 2011
    Messages:
    705
    Likes Received:
    194
    I've searched for some answers online in different forum. This might help:

     
  4. Andym

    Andym
    uix_expand uix_collapse
    Member

    Joined:
    Jul 22, 2010
    Messages:
    220
    Likes Received:
    11
    Thanks for your answers.
     
  5. lizatailor23

    lizatailor23
    uix_expand uix_collapse
    Member

    Joined:
    Mar 29, 2011
    Messages:
    73
    Likes Received:
    1
    Re:

    Subdomains are treated as a different sites. You can use the same code (Disallow: / ) but make sure it works with "subdomain . mywebsite . com/robots.txt" only and not with "www . mywebsite . com/robots.txt" etc., else you main site pages will also gets blocked.
     
  6. Andym

    Andym
    uix_expand uix_collapse
    Member

    Joined:
    Jul 22, 2010
    Messages:
    220
    Likes Received:
    11
    I have my robots.txt placed in the root of subdomain i.e: 'subdomain.mywebsite.com/robots.txt'. I also have the condition as 'Disallow: / and it is not helping me at all. However, a member on another forum has informed the following:

    Instead of:
    User-agent: *
    Disallow: /


    I must have
    User-agent: *
    Disallow: /http://subdomain.mywebsite.com
     
  7. goodbells

    goodbells
    uix_expand uix_collapse
    Member

    Joined:
    Dec 1, 2011
    Messages:
    48
    Likes Received:
    1
    USE This code for blocking sub domain :

    User-agent: *
    Disallow: /subdomainurl
     

Share This Page