Not a developer? Go to MovableType.com

News

Google Sitemap and Template Caching Tips

By Byrne Reese
Posted October 16, 2008, in Tips & Tricks.

On a recent call with the Movable Type Community we discussed and shared tips on how to best take advantage of Movable Type's built in caching system. One of the tips I shared is something I have been doing on my personal site and on a number of customer sites with great success, and it relates to how to optimize the publication of a Google Sitemap in a sustainable way.

Google Sitemaps have become an essential tool for bloggers to help optimize their content for search engines, also known as SEO or "Search Engine Optimization." Google Sitemaps are files published by a web site that detail each and every page on the web site and give some indication as to the frequency they are updated by the system. This helps ensure that when the Google fairy visits your site to slurp up and index your site's content that:

  1. Google indexes all of your content, and does not overlook any content that might be orphaned on your site because no other page links to it directly.

  2. Google is able to schedule additional visits of the Google Fairy when your content is likely to be updated.

Anyone can publish a Google Sitemap for the Movable Type powered web site or blog using a template provided by the community. This template has a number of parameters in it that can be used to fine tune your sitemap specifically for your web site. The challenge rests however in publishing this file. For some large sites, publishing a Google Sitemap can dramatically slow down publishing, as Movable Type needs to load every single entry and page into memory while it outputs a monstrous XML file. Some of these files have known to grow larger than several megabytes.

You can mitigate the impacts of publishing a sitemap by following these simple steps:

  1. Create a template module called "Google Sitemap Include" and paste into it the template provided by the community.

  2. For your "Google Sitemap Include" module turn on caching and set the cache to expire once every 24 hours.

  3. Now, create an index template called "Google Sitemap" and paste into its body the following template code:

    <mt:include module="Google Sitemap Include">
    

What this does is result in a Google Sitemap being published only once per day. This keeps your Google Sitemap up to date, while only incurring the cost of publishing it once. Handy. Of course this technique could be applied to virtually any template that can take a long time to publish, but also doesn't need to be up to date on a minute-by-minute basis.

We share a lot of tips on our community conference calls, you should join us next time!

Back

13 Comments

Michele Neylon

Michele Neylon on October 16, 2008, 4:33 p.m. Reply

Byrne

That’s an excellent tip - thanks for sharing it.

Now to implement it on all my blogs :)

Michele

Paul William Tenny

Paul William Tenny on October 17, 2008, 10:34 a.m. Reply

Can’t follow the link to the wiki because the wiki doesn’t load.

Daniel Stout

Daniel Stout on October 17, 2008, 12:31 p.m. Reply

Hey, Byrne — Is the Wiki site down? I tried early this morning, and again now (12:30 p.m. PDT). It is still not responding.

Dr. Mike Wendell

Dr. Mike Wendell on October 18, 2008, 1:59 p.m. Reply

Still down. :(

Have to admit that I just create my own with the following site. Let’s me double check broken links (And I’ve got a lot of them.) as I create the sitemap.

Dr. Mike Wendell

Dr. Mike Wendell on October 18, 2008, 2:01 p.m. Reply

Oops, forgot to give the link. Feel free to add the previous comment and remove this one. /me need coffee. :)

http://www.auditmypc.com/free-sitemap-generator.asp

Mike T

Mike T on October 24, 2008, 1:12 p.m. Reply

Byrne,

Your advice there about the caching is actually a good one for almost any index archive. I just wish that MT supported whole template caching, but the performance boost should be more than worth the cost of having a template module and a regular template for every index that you want to publish.

Mike T

Mike T on October 24, 2008, 1:19 p.m. Reply

By following this principle for most of my index templates, I was able to go from average rebuild time of all index templates of 23-30 seconds down to 8-9 seconds.

David Weiss

David Weiss on October 24, 2008, 8:01 p.m. Reply

I mentioned this on the page that contains the template for the Google sitemap, but I thought I’d post it here, too.

The sitemap does not seem to validate when using it with MT 4.2. It also does not create all your blog’s URLs properly. I re-wrote the template to conform with the updated MT tags and remove the deprecated tags. You can find the template code here:

http://www.daveweiss.net/2008/10/google-sitemap.html

Su

Su on August 13, 2009, 6:27 p.m. Reply

Alquma: That is completely untrue.

Google’s own information on sitemaps specifically states they “can’t guarantee that URLs from your Sitemap will be added to the Google index” at all.

freedom

freedom on October 13, 2009, 8:40 a.m. Reply

Has read all the ways. Thanks for councils. With sitemap indexation of pages occurs better. But also after ping pages get for a minute in google

alforque.myopenid.com

alforque.myopenid.com on November 17, 2009, 9:46 p.m. Reply

FYI: Google updated their Sitemap.

Here is the message in Webmaster tools —

November 13, 2009 Update your News Sitemaps to remain included in Google News

Recently, we introduced updates to the Google News Sitemap format to help you more easily manage your News Sitemap and articles. To avoid interrupting the inclusion of your articles in Google News, we will need your help.

Although we will support your current Google News Sitemap during the transition period of six months, you should re-submit it under the new format as soon as possible. For more details on how to submit your Sitemap using the new format (including how to add new tags to each entry in your Sitemap, to provide more information about individual articles), please visit the Sitemaps section of our Help Center at: http://www.google.com/support/news_pub/bin/topic.py?topic=11666.

Once the transition period is over, we will no longer accept News Sitemaps created using the old format and any old News Sitemaps in your Webmaster Tools account will be rejected.

We encourage you to make these changes as soon as possible in order to avoid interrupting your content’s inclusion in Google News. To get started, please visit our Help Center at: http://www.google.com/support/news_pub/bin/topic.py?topic=11666.

algerieinfo

algerieinfo on December 4, 2010, 1:11 a.m. Reply

What are all supported tags on google sitemap? I want to make a best structure mor my site http://www.monde-presse.com/

The sitemap.xml file seems to be get indexed very slowly.

Liam Stefanson

Liam Stefanson on July 19, 2012, 6:03 a.m. Reply

So, the MT is making it own sitemap and I don’t need to do anything right?

Byrne Reese

Byrne Reese was previously the Product Manager of Movable Type at Six Apart, where he had also held positions as the Manager of Platform Technology and Product Manager for TypePad. Byrne is a huge supporter of the Movable Type user and developer community. He dedicates much of his time to promoting and educating people about Movable Type as well as building the tools and plugins for Movable Type that are showcased on Majordojo. He contributes regularly to open source; and he is an advocate for open protocols and standards like Atom and OpenID.

Website: http://profile.typekey.com/byrnereese