Site URLs Change

Discussion in 'Search Engine Optimization' started by Fergal, Nov 13, 2010.

  1. Fergal

    Fergal
    uix_expand uix_collapse
    Premium Member
    Premium Member

    Joined:
    Nov 18, 2007
    Messages:
    10,578
    Likes Received:
    1,163
    We've recently updated the forum software that BAF runs on. One of the advantages of the update is that the new version of the software creates search engine friendly URLs. This has already become apparent in the site's SEO.

    The number of pages we have indexed in Google has almost doubled. I'm assuming that this is because our pages now have more than one URL. For example our Useful Online Business Resources thread can now be accessed from both of the URLs below;

    http://www.businessadviceforum.com/showthread.php/377-Useful-Online-Business-Resources
    http://www.businessadviceforum.com/showthread.php?t=377

    However, our search engine rankings for some of our primary keywords appear to have declined, following the update. Again I'm assuming that this is because individual pages each have different URLs.

    I feel that my choices are as follows;

    1. Discard the old URLs, maintain the new search engine friendly URLs and wait for the search engines to start ranking us highly again.
    2. Revert to the old "showthread.php?t=377" type URLs.

    Any opinions or advice would be greatly appreciated.

    Thanks!
     
  2. PaulPinnacle

    PaulPinnacle
    uix_expand uix_collapse
    Member

    Joined:
    Aug 24, 2010
    Messages:
    275
    Likes Received:
    59
    This is the case for the moment, but it's a temporary issue while the changes get sorted out on the SE side and propagate through their records. The 'new' URL's are being picked up and indexed, causing the dramatic increase in numbers, but at the same time the 301 (page permanently moved) is being flagged on the old pages (passing 'some', there is a deterioration in link juice through a redirect which can cause some issues for SERPs, or the old benefits and ensuring any old links or previously bookmarked pages are directed to the new location).

    If you try using the 'old' URL, you'll see that it is (correctly) 301 redirecting to the new 'friendly' URL. So there aren't any issues here in terms of duplicate content or user confusion from the change.

    This is pretty much already what is happening.

    The loss in ranking isn't really a surprise with such a dramatic change in the site URL's.

    Any thread which had a significant number of backlinks will have lost some of the benefit they once held due to the redirects now in place. This can be mitigated somewhat by reviewing the internal links pointing to the old threads and updating them to the new URLs and requesting external sites linking to the old URLs to update their links to the new friendly URLs, but that's a fairly huge task (perhaps consider a selective request on the external links to the webmasters you know will oblige - it's a better experience for their users to not get a 301 too).

    Personally, I'd keep a close eye on the crawling/indexing figures, give it a little more time to 'settle', update/gain new links to the friendly URLs and focus on the long term benefits of the changes (both for users and SEO) to offset the frustration as best you can.
     
  3. Fergal

    Fergal
    uix_expand uix_collapse
    Premium Member
    Premium Member

    Joined:
    Nov 18, 2007
    Messages:
    10,578
    Likes Received:
    1,163
    Paul, thanks very much for that, it is very reassuring. I hadn't noticed that the old URLs were being re-directed to the new ones, foolish me. I'll schedule some time next week to fix as many of the internal links as I can find.
     
  4. PaulPinnacle

    PaulPinnacle
    uix_expand uix_collapse
    Member

    Joined:
    Aug 24, 2010
    Messages:
    275
    Likes Received:
    59
    I didn't think to check at first. I was too busy thinking 'Poor Fergal' and about to launch into a description of some fairly nasty (in terms of getting it right) htaccess rewrites and redirects to resolve the duplicate content issues. I just happened to do a quick check on how vB handles the friendly URL's which showed that they do much of the work by default and confirmed it was the case for BAF. Saves you a fairly nasty headache, so great news.

    I'd normally suggest, in cases like this, to keep the focus firmly on the external links (they carry more weight). However, given the relatively low number of deep links to BAF at the moment (something to focus on in future, promoting some/many of the great resource/category pages - a very high proportion of the backlinks focus on the homepage at the moment) and the huge number of internal links, in this case it's worth trying to address both (as far as is practical or possible).
     
  5. Fergal

    Fergal
    uix_expand uix_collapse
    Premium Member
    Premium Member

    Joined:
    Nov 18, 2007
    Messages:
    10,578
    Likes Received:
    1,163
    Thanks again Paul, finding the majority of those internal links shouldn't be too difficult with the help of the "Search within a site or domain" feature on Google Advanced Search.
     
  6. PaulPinnacle

    PaulPinnacle
    uix_expand uix_collapse
    Member

    Joined:
    Aug 24, 2010
    Messages:
    275
    Likes Received:
    59
    While using the advanced search queries will help to identify 'some', it's not ideal for this situation (those advanced queries are great for investigating things like your link profile, where what is indexed is part of the equation, but not ideal for resolving site issues where you want a more holistic approach that is independent of indexing/crawl depth). For 'site' checks like this, something like 'xenu link sleuth' or 'IIS7' would be preferable to identify 'all' cases, rather than a 'selection'. (Caveat, for a site as large as BAF, you'll probably need to restrict the tools slightly [e.g. <10 levels deep] - or at least take care that it might crash your system as it approaches very large number [I know I hit +850,000 for BAF on xenu before, it didn't crash on me, but a trip to hospital at the time meant that I ended up losing the data])

    Worth having a chat with CGI Web prior to any changes. His knowledge might allow for many of the updates to be scripted/automated, saving a significant amount of time in the process.

    Slightly off topic, but worth noting that as some of the site speed issues get addressed (as per the other thread) and some tweaks made to optimize same, the level/number of pages indexed will/should also increase. Given the level of traffic that BAF can achieve through long tail queries (thus directly linked to the number of pages indexed), something to give plenty of attention to. With that in mind, going a little beyond the indexed links should end up with benefits (but there will be a point of diminishing return, so a balance to be struck).
     
  7. Mike.Zana

    Mike.Zana
    uix_expand uix_collapse
    Banned

    Joined:
    Nov 14, 2010
    Messages:
    8
    Likes Received:
    0
    I think you should be patient and wait for the search engines to start picking up the new URL's, it will be worth it in the end.
     
  8. Fergal

    Fergal
    uix_expand uix_collapse
    Premium Member
    Premium Member

    Joined:
    Nov 18, 2007
    Messages:
    10,578
    Likes Received:
    1,163
    Thanks Paul, I will do an initial search for the link titles in Google advanced search and then look at the tools you suggest.

    Thanks also Mike.Zana, My original concern was that we may have had duplicate URLs for the same content and that the search engines might penalise us for that. Now that Paul has pointed out that the 301 redirects have been created and that our URL structure is not damaging our search engine listings in the longer term, I'm happy to wait.
     
  9. John Million

    John Million
    uix_expand uix_collapse
    Member

    Joined:
    Oct 22, 2010
    Messages:
    56
    Likes Received:
    2
    Two quick points: redirects are definitely the way to go to maximize your chances of preserving SEO value. If one URL is just a redirect to the new one, you shouldn't get a duplicate content penalty because Google will notice that you made a big change all at once, and added redirects to help Google's crawlers AND people who previously had certain content bookmarked. Don't change your structure now, Google takes up to a month to crawl some pages. Give it time to adapt to the new structure before you change it again. Second, I haven't dug around too much so I might be missing it, but I'm not seeing a sitemap. Google's crawlers DO NOT crawl every page at the same rate, and in some circumstances they randomize their link choices. If you have a page with 200 links (like a forum) the crawler will check the first 10, then maybe 12, 15, 19... and finally just start checking every 10th link. An easy-for-crawlers-to-find sitemap acts as a cheat-sheet so the Googlebot can understand your site structure (and the likely relevance of each page, SEO) much easier.
     
  10. Fergal

    Fergal
    uix_expand uix_collapse
    Premium Member
    Premium Member

    Joined:
    Nov 18, 2007
    Messages:
    10,578
    Likes Received:
    1,163
    Thanks for the suggestions and advice John Million. A site map is something that I definitely need to get sorted and will do it in the near future.
     

Share This Page