Skip to main content

What Is URL Canonicalization (URL Normalization)? And What Are Its Effects on SEO?

Have you ever noticed that when you type in www.twitter.com in the address bar, after Twitter loads, it will be changed to http://twitter.com? In the same way, if you type in bbc.co.uk in address bar, after BBC gets loaded, the URL will be http://www.bbc.co.uk. If you try to browse to microsoft.com you will get an entirely different URL in the form: "http://www.microsoft.com/en/us/default.aspx". You find that certain websites strip off your 'www' prefix, while certain others add it. Yet another set of websites redirect you to entirely different looking URLs.

In this post, we will see more about this aspect of search engine optimization, called URL canonicalization or URL normalization. We will see why it is important and why being knowledgeable about it will reduce duplicate content issues.

Need for Canonicalization

As you have seen above, the following URLs may all be the same in terms of content:
http://website.com
http://www.website.com/index.html
http://www.website.com/default.aspx
http://website.com/default.aspx


But they are different URLs in the eyes of search bots. Which means, search engines will crawl them independently looking for different content, while ending up fetching the same content. It causes your site be crawled four times for the same content. You can imagine the load it will cause on your server. On the top of that, it will cause your website be recognized as a copy domain. Another effect is that all of these four variants will have different PageRank in Google index.

This is why we need to do canonicalization of URLs, which is standardizing a general URL structure for the entire domain.

Steps to Canonicalize URLs

Here are the steps you follow to canonicalize your URLs.

1. Choose WWW or Non-WWW form

First step is of course choosing whether you want WWW version or non-WWW version for your URLs. If you are using a self-hosted blog, then you can choose whichever you like. On the other hand, if you are hosting your blog on Blogger or WordPress free domain or TypePad, then the non-WWW version is what you need to choose. All the URLs in these services follow the non-WWW version by default and you do not have access to the server to change it.

The first step is to set up a 301 redirect from the non-preferred version to the preferred one. Setting up 301 redirect is beyond the scope of this post; for a detailed description for various servers, please see here. However, this is not important in case of Blogger or WordPress, since you don't have server-level access.

Now, you have to choose the preferred version at Google Webmaster Tools (not important if you have set the 301 redirect). Log into Webmaster tools with your Google account, Go to Settings [in the latest changes, settings holds most of the settings], and choose the preferred domain as whichever version you have chosen. This is enough for your Blogger/WordPress domain. Sadly, this feature is not available yet in Yahoo or Live search.

2. Use the Canonical URL Everywhere in the Domain

Now, you have to show the search engines that you prefer one version of the URL to the other. For this purpose, you need to use the preferred version across your domain, and recommend other people to follow it. For instance, if you check my internal links, you will see that I always strip off the 'www' part. I recommend other people that link to my blog to strip off the 'www' part as well. This way, I can be sure that search engines will pick up the preferred version and display it.

3. Remove Default File Names From URLs

Depending on your web server, you will have default file names (like 'index.html' or 'index.shtml' in Apache server and 'default.aspx' in Microsoft IIS), which means, when a visitor comes to your site's directory, he will be actually seeing "http://website.com/default.aspx".

You need to change this aspect and strip off the file name from the URL.

4. Minor Canonicalization Issues

There are more to canonicalization of URLs, such as keeping URLs entirely in lower case, keeping or not keeping the slashes at the end (http://website.com/ vs. http://website.com), removing fragments (#title), etc. More on this can be read in the paper by Tim Berners-Lee (the founder of world wide web) and others, published at the Internet Engineering Task Force (IETF): http://tools.ietf.org/html/rfc3986.

Conclusion

Make sure you read the related resources section below to know more on this. Keep a single URL structure across your domain, and encourage anyone linking your site to follow this. This way, you will keep your website search-engine-friendly.

Related Resources

URL canonicalization FAQ at MattCutts.com
Setting up 301 redirects on various servers

Copyright © Lenin Nair 2008

Comments

  1. Very informative. Keep it up. Thanks.

    ReplyDelete
  2. Very creative and educating.
    How about sites, many many, that fail to load if you dont type www?--I mean just try http://heartstroke.ca/ (a north American sample) for test and try a site from any site that is outside india.com/ (a South Asian sample)
    Best wishes.

    ReplyDelete
  3. Thanks for your comment. Both your site examples worked for me without www.

    ReplyDelete

Post a Comment

Comments are moderated very strictly

Popular posts from this blog

En Dash, Em Dash, and Hyphen

We have three types of dashes in use: The hyphen, En Dash, and the Em Dash. In this post, we will see how to use them all correctly. Hyphen (-) The hyphen is the minus key in Windows-based keyboards. This is a widely used punctuation mark. Hyphen should not be mistaken for a dash . Dash is different and has different function than a hyphen. A hyphen is used to separate the words in a compound adjective, verb, or adverb. For instance: The T-rex has a movement-based vision. My blog is blogger-powered. John’s idea was pooh-poohed. The hyphen can be used generally for all kinds of wordbreaks . En Dash (–) En Dash gets its name from its length. It is one ‘N’ long (En is a typographical unit that is almost as wide as 'N'). En Dash is used to express a range of values or a distance: People of age 55–80 are more prone to hypertension. Delhi–Sidney flight was late by three hours. In MS Word, you can put an En Dash either from the menu, clicking Insert->Symbol or by the k

4 Effective Ways to Write About a Boring Topic

  With the plethora of interesting topics to write about, you’re fortunate enough to get the “boring” one. While it can be a pain for many writers to wind up with such a task, I’m telling you now there are ways to make yours more interesting than it is. So if you find yourself stuck with the dreariest topic to fill in a blog about, don’t fret. Here are the four best ways to unburden yourself. 1. Never a boring topic, only a boring writer. Here’s the hard fact: It’s never about the topic being boring. It’s about the writer making it boring. For instance, you’re supposed to write about aquariums. I know, how can you continuously make this topic interesting, right? Well, you’d be surprised just in how many ways you can make it an enticing read. Start by listing down the basic “what”, “where”, “when” and “how” surrounding the topic. You can ask (and research) about “What material was first used to make aquariums?” or even “How the first aquarium was built?” or “What are

5 Simple Ways To Improve Your Writing

As writers, we’re always pushing the envelope to express ourselves better than we did before. However there are times when we plateau, when our writing just doesn’t feel natural. Worse yet, there are times when we can’t get a good paragraph out. Those are times when we start thinking about pushing ourselves and our writing. When you need to tweak and boost your writing up a notch, it’s always good to try something constructive.  For those who could use a couple of tips, here are a few that are sure to help improve your writing and keep it from going stale. 1.Imitate Different Authors If you read other writers for inspiration, why not actively imitate their writing? Once you walk in another writer’s shoes for a bit, writing as they write, you get an idea on how to approach writing about things in ways you normally wouldn’t.  Moreover, it will force you to pay attention to what makes someone else’s writing style unique which, in turn, will help you find ways to make your