Skip to main content

What Is URL Canonicalization (URL Normalization)? And What Are Its Effects on SEO?

Have you ever noticed that when you type in www.twitter.com in the address bar, after Twitter loads, it will be changed to http://twitter.com? In the same way, if you type in bbc.co.uk in address bar, after BBC gets loaded, the URL will be http://www.bbc.co.uk. If you try to browse to microsoft.com you will get an entirely different URL in the form: "http://www.microsoft.com/en/us/default.aspx". You find that certain websites strip off your 'www' prefix, while certain others add it. Yet another set of websites redirect you to entirely different looking URLs.

In this post, we will see more about this aspect of search engine optimization, called URL canonicalization or URL normalization. We will see why it is important and why being knowledgeable about it will reduce duplicate content issues.

Need for Canonicalization

As you have seen above, the following URLs may all be the same in terms of content:
http://website.com
http://www.website.com/index.html
http://www.website.com/default.aspx
http://website.com/default.aspx


But they are different URLs in the eyes of search bots. Which means, search engines will crawl them independently looking for different content, while ending up fetching the same content. It causes your site be crawled four times for the same content. You can imagine the load it will cause on your server. On the top of that, it will cause your website be recognized as a copy domain. Another effect is that all of these four variants will have different PageRank in Google index.

This is why we need to do canonicalization of URLs, which is standardizing a general URL structure for the entire domain.

Steps to Canonicalize URLs

Here are the steps you follow to canonicalize your URLs.

1. Choose WWW or Non-WWW form

First step is of course choosing whether you want WWW version or non-WWW version for your URLs. If you are using a self-hosted blog, then you can choose whichever you like. On the other hand, if you are hosting your blog on Blogger or WordPress free domain or TypePad, then the non-WWW version is what you need to choose. All the URLs in these services follow the non-WWW version by default and you do not have access to the server to change it.

The first step is to set up a 301 redirect from the non-preferred version to the preferred one. Setting up 301 redirect is beyond the scope of this post; for a detailed description for various servers, please see here. However, this is not important in case of Blogger or WordPress, since you don't have server-level access.

Now, you have to choose the preferred version at Google Webmaster Tools (not important if you have set the 301 redirect). Log into Webmaster tools with your Google account, Go to Settings [in the latest changes, settings holds most of the settings], and choose the preferred domain as whichever version you have chosen. This is enough for your Blogger/WordPress domain. Sadly, this feature is not available yet in Yahoo or Live search.

2. Use the Canonical URL Everywhere in the Domain

Now, you have to show the search engines that you prefer one version of the URL to the other. For this purpose, you need to use the preferred version across your domain, and recommend other people to follow it. For instance, if you check my internal links, you will see that I always strip off the 'www' part. I recommend other people that link to my blog to strip off the 'www' part as well. This way, I can be sure that search engines will pick up the preferred version and display it.

3. Remove Default File Names From URLs

Depending on your web server, you will have default file names (like 'index.html' or 'index.shtml' in Apache server and 'default.aspx' in Microsoft IIS), which means, when a visitor comes to your site's directory, he will be actually seeing "http://website.com/default.aspx".

You need to change this aspect and strip off the file name from the URL.

4. Minor Canonicalization Issues

There are more to canonicalization of URLs, such as keeping URLs entirely in lower case, keeping or not keeping the slashes at the end (http://website.com/ vs. http://website.com), removing fragments (#title), etc. More on this can be read in the paper by Tim Berners-Lee (the founder of world wide web) and others, published at the Internet Engineering Task Force (IETF): http://tools.ietf.org/html/rfc3986.

Conclusion

Make sure you read the related resources section below to know more on this. Keep a single URL structure across your domain, and encourage anyone linking your site to follow this. This way, you will keep your website search-engine-friendly.

Related Resources

URL canonicalization FAQ at MattCutts.com
Setting up 301 redirects on various servers

Copyright © Lenin Nair 2008

Comments

  1. Very informative. Keep it up. Thanks.

    ReplyDelete
  2. Very creative and educating.
    How about sites, many many, that fail to load if you dont type www?--I mean just try http://heartstroke.ca/ (a north American sample) for test and try a site from any site that is outside india.com/ (a South Asian sample)
    Best wishes.

    ReplyDelete
  3. Thanks for your comment. Both your site examples worked for me without www.

    ReplyDelete

Post a Comment

Comments are moderated very strictly

Popular posts from this blog

What Is the Difference Between Hardcover and Paperback?

Today, my reader, Rahman contacted me with a doubt:

Dear Lenin, would you explain why there are two types of books: hardcover and paperback?
This is quite a simple affair and there are explanatory articles to be found at various places on the Net. Here is my addition.

Hardcover

A hardcover aka hardback is a book bound with thick protective cover, with usually a paper or leather dust jacket over the main cover. The aim of hardcover is protection and durability. These books are mainly for long-term use and collectors’ editions. Hardcover books last far longer than the corresponding paperbacks. They do not get damaged easily thus making them perfect for reference guides, great literary works, etc.

In addition, there is a difference in the type of paper used to print hardcover books. The paper used is long-lasting acid-free type. Acid-free paper has a pH value of 7 (neutral) which makes it highly durable. The papers are stitched and glued to the spine.

Hardbacks are prepared for commercial …

En Dash, Em Dash, and Hyphen

We have three types of dashes in use: The hyphen, En Dash, and the Em Dash. In this post, we will see how to use them all correctly.

Hyphen (-)

The hyphen is the minus key in Windows-based keyboards. This is a widely used punctuation mark. Hyphen should not be mistaken for a dash. Dash is different and has different function than a hyphen.

A hyphen is used to separate the words in a compound adjective, verb, or adverb. For instance:

The T-rex has a movement-based vision.
My blog is blogger-powered.
John’s idea was pooh-poohed.


The hyphen can be used generally for all kinds of wordbreaks.

En Dash (–)

En Dash gets its name from its length. It is one ‘N’ long (En is a typographical unit that is almost as wide as 'N'). En Dash is used to express a range of values or a distance:

People of age 55–80 are more prone to hypertension.
Delhi–Sidney flight was late by three hours.


In MS Word, you can put an En Dash either from the menu, clicking Insert->Symbol or by the key-combination, Ctrl + Num…

What Is the Meaning of the Word 'Ghajini'? Story and Trivia of Aamir Khan's New Film [Special]

[Special Entry]



Aamir Khan's latest film is titled a little weirdly for the taste of Hindi filmgoers. 'Ghajini': They have never heard of such a name, and such a word never existed in Hindi or in any other Indian language.

The name Ghajini is the name of the villain of the film. In Tamil version, the name of the villain was Laxman.

As a Tamil moviegoer, I have already watched Ghajini and know the story in full.

So, What Does the Title Mean?

In Tamil, the title of the film is inspired by the story of Mahmud of Ghazni, an ancient invader of India. This person was so persistent in invading India that he continued trying after several failures. In the film too, the protagonist is such persistent in finding out and killing the villain of the film, who had killed his girlfriend, Kalpana (played by Asin). Aamir's Character (named Sanjay Ramaswamy in Tamil), is a short-term amnesiac, who cannot remember anything more than fifteen minutes.

You may ask then how the Ghazni became…