Ultimate Guide to Template Module Caching

Movable Type offers a number of features to help administrators increase the performance of their system. Chief among them is template module caching. As the name may imply, any template module in Movable Type can have caching enabled, which causes Movable Type to store the output and HTML generated by processing the contents of the module in Movable Type's database for later use. That way, if a template module is used frequently, then Movable Type need only process the contents of the module once.

The following guide will give you an in-depth tutorial on how to use this important feature of Movable Type. It will also instruct you on techniques you can employ to dramatically speed up publishing on your web site.

Why should I use template caching?

Publish more, faster

The first thing most users notice after enabling caching for a number of key template modules is that the time it takes to publish pages on their site goes way down. That is because the primary benefit of this feature is to reduce the work and time it takes to render the contents of a page.
Increase system and database performance and reliability

With module caching enabled across your site, there is another side benefit you receive closely correlated to performance: decreased load on your database. When caching is enabled, and Movable Type no longer needs to parse, compile and render the contents of a template module, the number of database queries made by Movable Type during the publishing process is greatly reduced.

Why does this matter? Well, a database is a very complex application that requires a lot of memory, file system access and CPU. The more queries a database needs to process the more of an impact it can have on one or more of these critical resources. And when any one of these resources comes under high demand within a system, the more likely the entire system can slow down. Think for a moment abut all of the other applications on your server that also require these resources: like processing email, serving pages on your web server and responding to requests sent to the Movable Type application itself for example. When the database monopolizes these resources, then not only does Movable Type slow down, but everything on the system slows down.

It is for this compounding effect that the database has upon overall system performance that template module caching can offer such great performance benefits. When template module caching is employed properly not only can publishing performance increase, but performance across the entire system can increase.

How does template caching work?

Let's first talk a little bit about how caching works in general. Knowing this will help you to take better advantage of this feature within Movable Type, as well as to help you understand some of the limits of what it can and cannot do.

What is a cache?

A "cache" is merely a system that makes frequently used content available in such a way that it is easier, faster and more efficient to retrieve that content from the cache, than it is to recompute or regenerate it from scratch. A cache is also responsible for managing how long an item in the cache will stay on the cache before it expires. When content on a cache expires, it is physically removed from the cache to ensure that the next time the system asks to retrieve that specific content it will be forced to regenerate it to ensure that it is up to date.

Another important component of a cache is that it operates within a system invisibly. In other words, when a system utilizes a cache, it will interface with the cache without the user ever having to be aware of the fact. In this way the user never needs to worry or think about the cache and its contents - it should just work.

Template Module Caching

In the process of rendering a template and processing an <mt:Include> tag Movable Type will go through the following steps:

In so doing, Movable Type will either compute the contents of a module and return them, or pull the module's contents from the cache.

Using Template Module Caching

To begin, you will first need to enable "Module Caching" on the Preferences > Publishing Settings screen for each blog that will utilize this feature. To do so, click the check box labeled "Module Caching" under the "Module Options" section. Then click "Save Settings."

Cache Expiration

Every template module that has caching enabled will also have its own unique expiration preferences. There are two primary ways one can configure when a module's contents will expire on the cache:

after a designated time period, e.g. "keep this module's contents on the cache for a maximum of 20 minutes."
in response to a specific event on the system, e.g. "keep this module's content on the cache until someone creates a new entry."

The expiration preferences you choose will depend greatly upon the content contained by the module. Let's explore a few different scenarios and template modules and discuss which expiration preference might make the most sense for that module.

Recent Entries Widget

The "Recent Entries" widget is an element commonly found in the sidebar of every page of a blog or web site. The purpose of this module is to display to your readers other recent blog posts they might be interested in. Given that the contents of this module only change when a new entry is created within the system, then it makes sense to expire this widget/module's contents when a new entry is created.

Recent Comments Widget

Similar to the previous example, the "Recent Comments" widget is the same across every page of your web site or blog, and its contents are only updated when a new comment has been received. Therefore, the expiration preferences that make the most sense for this widget would be to expire the widget when a new comment is created.

RSS Widget

Suppose you are using Feeds.App Lite to aggregate headlines from your favorite blog elsewhere on the Internet. Going out over the Internet to fetch this blog's feed, to parse the XML, and then render the feed's contents in your module can be an expensive proposition - especially if you want to display this module's contents on every page of your blog.

Given that there is no event by which you can logically expire this module's contents, it might make more sense to utilize a time-based mechanism for content expiration. Therefore you need to ask yourself, "how often will this RSS feed likely be updated on the remote web site?" Suppose for example the blog you are aggregating headlines from updates only daily, then a logical cache period might be 12 or 24 hours. It is however important to point out, the regardless of the frequency this RSS feed gets updated, setting the cache to anything at all will reduce the number of times Movable Type has to go and check, thereby increasing performance some percentage of the time - therefore setting the cache to even 1 hour will have substantial benefits for your site.

Other Examples

The basic examples above should get you started in understanding how template module cache expiration works. Later in this guide we will explore more advanced techniques that will help you fine tune your cache and maximize the benefit of using this feature.

Determining what to cache

This is million dollar question as the answer to this will depend largely upon your own unique setup and design. Even so, there are some very common widgets and modules that most users can enable caching for that would have near immediate benefits. They are:

Recent Entries - set to expire upon the creation of an entry.
Recent Comments - set to expire upon the creation of a comment.
Monthly Archives - set to expire once every 24 hours.
Category Archives - set to expire upon the creation of a category, and optionally upon the creation of an entry (if you display entry counts for each category)
Recent Assets/Photos, or Gallery - set to expire upon the creation of a new asset.

Tip: Utilizing a "5-minute cache"

One trick you can employ for virtually any module or widget whose content is shared across every page in the site, and whose content is updated frequently enough to warrant it not being cached at all is a very short lived cache, e.g. 2-5 minutes. This has the benefit of caching the module's content long enough to only compute it once per publishing session, but not long enough that it is likely to get updated and refreshed in time for the next publishing session.

Movable Type 4.0 and 4.1 Users

In Movable Type 4.0 and 4.1, the structure of Movable Type's default templates were such that all of a blog's widgets were hard coded directly within the "Sidebar" template module. In this format, it is impossible to set up individual caching rules for each of the various widgets that could appear on your web site.

If you are Movable Type user and see a "Sidebar" module listed among your templates, consider segmenting the sidebar into multiple modules/widgets and then caching each one individually.

Clearing the Cache for a Module

Once a module's output has been cached, there may be circumstances in which you wish to force an update to the cache in order to keep your web site's content fresh and up-to-date. For example, suppose you set the cache to expire after 24 hours, but you don't want to wait 24 hours for the cached content on your web site to be updated.

To force Movable Type to expire the contents of a cache associated with a template module, all you need to do is edit and then save that template module. You don't need to make any changes to the template in the process. The act of saving a template forces the cache for that module to be invalidated because the contents of the module could have been altered dramatically.

Server Side Includes and Module Caching

A feature commonly confused with template module caching is "server side includes." Server side includes, or "SSI," relate to how the contents of the module are included in the page, where as template module caching relates to frequency with which the contents of a module are computed and refreshed.

Still confused? Let's illustrate with an example. First, let's look how Movable Type treats the same template code with SSI turned off and on.

First, let's assume our system has a template module called "My Module" with the following code:

<mt:entries lastn="2">
  <p><mt:entrytitle></p>
</mt:entries>

Now, another one of our templates has the following template code:

<h2>The last 2 entries for your blog are:</h2>
<mt:include module="My Module">
<p>Thank you.</p>

With SSI Turned Off

With server side includes turned off for "My Module," the above code is functionally equivalent to the following code sample where the <mt:include> tag is simply replaced with the contents of the "My Module" module, like so:

<h2>The last 2 entries for your blog are:</h2>
<mt:entries lastn="2">
  <p><mt:entrytitle></p>
</mt:entries>
<p>Thank you.</p>

With SSI Turned On

With server side includes turned on for "My Module" thats when things get really interesting. When Movable Type processes the original template sample and encounters the <mt:include> tag, it will actually output the following:

<h2>The last 2 entries for your blog are:</h2>
<?php include('/path/to/blog/includes_c/my_module.php');
<p>Thank you.</p>

Movable Type will then immediately process the contents of "My Module" and output its contents to your web server's file system, in your blog's includes_c directory (a directory created and maintained for you automatically by Movable Type).

The Benefit of Server Side Includes

The core benefit of using server side includes lies in the fact that when multiple pages include the same template module, updating one of those pages will result in instantly updating all of the pages that include that module.

Leveraging Template Module Caching

Turning on server side includes for a module has great benefits, but it does not by itself offer any performance benefits. Consider the following scenario:

you have template module caching disabled
you have server side includes enabled for your "Recent Entries" widget

In this example, the contents of the "Recent Entries" widget will be recomputed and rewritten to your site's includes_c directory for each and every page of your site that is published, needlessly. It is needless because with SSI enabled, the list of recent entries produced by the "Recent Entries" widget will be updated instantly across your entire web site after the first page is published.

Now, let's reconsider the above scenario, but with template module caching enabled. In this case, the contents of the "Recent Entries" widget is only computed and written to disk once, while still being updating instantly across every page of your web site that includes this widget.

In other words, if your system supports server side includes it is strongly encouraged you use them in conjunction with template module caching. When used together, not only do you get the performance benefits provided by caching, but your entire site will stay up to date with less need to republish.

Things to consider about caching

Cache Scope or Context

By default, Movable Type stores each module's cached content once per blog. Take for example a template module called "My Module" with the following contents:

Hello <mt:var name="hello">.

Now, let's say that module is included three times in a single template like so:

<mt:var name="hello" value="Chewbacca">
<mt:include module="My Module">
<mt:var name="hello" value="Han Solo">
<mt:include module="My Module">
<mt:var name="hello" value="Wedge Antilles">
<mt:include module="My Module">

With caching turned off, this template would output the following:

Hello Chewbacca.
Hello Han Solo.
Hello Wedge Antilles.

With caching turned on, this template would produce:

Hello Chewbacca.
Hello Chewbacca.
Hello Chewbacca.

Why is this? Well, the output from "My Module" depends upon the context in which it is included, namely, the value of the "hello" variable.

To better understand this concept of a global scope let's discuss in greater detail how Movable Type stores and then subsequently looks up contents in the cache.

Each and every item in the cache for a given blog is assigned a "cache key." This cache key is what is used by Movable Type to store and retrieve items from the cache. A template cache key is typically determined by "dirifying" the template module's name. Using this system, the cache key for "Recent Entries" would be "recent_entries" and the cache key for a module called "This blog's authors" would be "this_blogs_authors."

Since this system bases the cache key upon the name of the module being included, you can see how regardless of where the template module "My Module" is included, it will always refer to the same item in the cache because it will always resolve to the same cache key: "my_module."

Most of the time.

We say "most of the time" because there is a way you can designate your own cache key when storing and looking up items in the cache. This is done by utilizing a special key argument you can pass to the <mt:include> tag. When using this argument, you can override the default cache key and designate your own. For example:

<mt:SetVarBlock name="cache_key"><mt:EntryCategory></mt:SetVarBlock>
<mt:Include widget="Related Entries" key="$cache_key">

In the example above, a cache key is created that is based upon the name of the current entry's primary category, thereby permitting entries that belong to the "News" category to point to one item in the cache and entries that belong to the "Entertainment" category to point to another.

Output vs. Evaluation

It is important to understand that Movable Type will only cache the output from a template. If a module is responsible for setting template variables that subsequent templates rely upon, then caching should not be used for that template. For example, consider a template module called "My Module" with the following contents:

<mt:var name="foo" value="123">

Now consider a template that includes that module like so:

<mt:include module="My Module">
foo = <mt:var name="foo">

If "My Module" had caching enabled then Movable Type would output the following content the first time the template was published:

foo = 123

The second time it was published, Movable Type would output the following:

foo =

This is because when Movable Type processes the <mt:include> tag the second time around, it retrieves the module's content from the cache. And because the <mt:var> tag does not produce any output the cache will contain an empty string.

In the above sample, because "My Module" contains code that must be evaluated, it is essential that caching for that module be disabled.

Nested Modules

When modules are nested within Movable Type, meaning when Module A includes Module B, and Module B includes Module C, then the outermost include with caching enabled supercedes all the rest. For example, let suppose:

Module A has a cache expiration of 24 hours
Module B is included by A and has a cache expiration of 1 hour.

In this scenario, even if the contents of Module B are likely to be updated every 1 hour, you will only see it get updated once every 24 hours because the contents of Module A, which include the contents of Module B, will only be refreshed at that interval.

Global Template Modules

Template module caching is not supported for global template modules. This is unfortunate, but it can easily be worked around using the following technique:

Create a global template module called "My Global Module Foo."
In Blog A, create another template Module called "My Module Foo." Set this module's contents to:
```
<mt:include module="My Global Module Foo">
```
Set the caching preferences you desire for "My Module Foo."

This technique allows you to share design elements between blogs, while allowing each blog to define its own caching preferences for that module.

Memcached Support

One nice feature of Movable Type's template caching system is that if you have memcached enabled, template module content will be cached there instead of the database. This is important for systems whose databases are extraordinarily busy even with module caching enabled.

Advanced Caching: Using the tag

The <mt:Include> template tag supports a number of arguments that give designers and administrators even more fine grained control over their caching preferences associated with a template module, than what can be expressed using Movable Type's user interface.

These arguments are:

cache - if set to '0' this will forcibly turn off caching for this module. If set to '1' then caching will be turned on.
ttl - sets the "time to live" in seconds for the cached content
cache_key - sets the lookup key for the module's cached content.

Example

The following will cache a version of the "Related Entries" widget on a category-by-category basis.

<mt:SetVarBlock name="cache_key"><mt:EntryCategory></mt:SetVarBlock>
<mt:Include widget="Related Entries" key="$cache_key">

Frequently Asked Questions

Are there any prerequisites for template module caching?

Template module caching will work for any Movable Type installation. The use of memcache is optional. If memcache is enabled, Movable Type will store cached content there in lieu of in the database, further reducing database load.

Why would I want to use template module caching?

Template module caching can dramatically speed up publishing and reduce database load. Caching is recommended for any and all sites of scale that require a lot of publishing.

What modules and widgets do you recommend that I cache?

Almost all widgets could benefit in some way from caching. For example, you can set your "Recent Entries" widget to be cached, and have that cache live for as long as it needs to until a new entry is received, and that widget needs to be updated.
Don't cache modules that are trivially small, or do not contain any Movable Type template tags.
Focus your efforts around finding blocks of code that potentially utilize complex template code. Look specifically for template tag combinations that you might suspect slow down the publishing process, and then place that code inside of a cachable template module.

Why would I set caching preferences inside of the MTInclude template tag?

The template modules options in the user interface represent a simple and flexible way to achieve the vast majority of caching scenarios. However, in the event that you need even more control over how and when things get cached, you can explore a whole new set of arguments added to the <mt:Include> tag.

For example, suppose you have a template module that is shared across a number of different templates. And suppose the output from that template module will vary greatly depending upon the template in which it is used. Therefore a simple global cache for that module is inadequate.

The key argument in the <mt:Include> tag will allow you to specify a lookup key for the cached content that is specific to the context in which it is used, thus allowing a single template module to cache 5 or more different versions of itself.

What is "TTL?"

"TTL" stands for "Time To Live" and refers often to the amount of time given to an item in the cache before it expires and must refreshed.

The content on my site is not updating even though I changed a template, how do I fix this?"

It is quite possible that one of your templates is invoking a template module that has been cached. To clear the cache, find the template being cached, and save it. Saving it, even if you don't make any changes to it, will force MT to invalidate and clear its cache.

You can also navigate to that blog's Publishing Settings' screen and turn off template module caching across the board.

How would I cache the contents of an entire page?

In the event that you want to cache the contents of an entire page or index template, place the entire contents of the page inside a module and have the index template simply include that module.