Setting up Movable Type with Memcached
Simply put, memcached (pronounced mem-cash-dee) is a caching framework that allieviates database load and makes applications that utilize it faster and more reliable. From memcached's website:
memcached is a high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.
Danga Interactive developed memcached to enhance the speed of LiveJournal.com, a site which was already doing 20 million+ dynamic page views per day for 1 million users with a bunch of webservers and a bunch of database servers. memcached dropped the database load to almost nothing, yielding faster page load times for users, better resource utilization, and faster access to the databases on a memcache miss.
How to Install Memcached in Unix
Some RPMs exist for memcache, but the most reliable way to get memcached up and running is to compile from source. This is typically very straight forward and quite reliable.
- Download libevent (version 1.3b).
- Untar libevent by executing tar zxvf libevent-1.3b.tar.gz
- Execute cd libevent-1.3b
- Execute ./configure
- Execute make install as root
- Download memcached (version 1.2.2).
- Untar memcached by executing tar zxvf memcached-1.2.2.tar.gz
- Execute cd memcached-1.2.2
- Execute ./configure
- Execute make install as root
./memcached -d -m 2048 -l 127.0.0.1 -p 11211 &
This will start memcached and bind it to port 11211 on the IP address of 127.0.0.1 with 2GB of memory.
How to Install Memcached in WindowsA version of memcached exists for windows as well. Download the latest Win32 Binary of memcached and follow the instructions found on its website.
How to Install Memcached on Mac OS X
To install memcached on a Mac OS X system, download and run the following script.
Setting Up Movable Type
Once memcache has been successfully installed on your machine, or on a server that is accessible by Movable Type, edit your mt-config.cgi file and add the following configuration parameter:
MemcachedServers 127.0.0.1:11211Be sure to set the IP address (127.0.0.1 in the example above) and the port number (11211 in the example above) to the proper IP and Port that memcached is listening on.
To setup Movable Type to connect to a pool of memcache servers add multiple MemcachedServers directives like so:
MemcachedServers 127.0.0.1:11211If you are running Movable Type under FastCGI you will need to restart your webserver.
MemcachedServers 127.0.0.2:11211
MemcachedServers 127.0.0.3:11211
That's it. Movable Type will immediately begin caching data using memcached. If Movable Type experiences any problems accessing memcached then it will degrade gracefully and act as if it is not there.
Tips and Best Practices
- Consider launching memcached automatically when the server starts up so that you can rest assured that it will always be running.
- Monitor memcache using Nagios or other monitoring software to ensure that it is running
- Consider deploying memcache on a dedicated machine
- Do not deploy memcached and Movable Type on different networks - memcache operates most efficiently when network overhead and latency is completely minimized.
18 Comments
Byrne Reese
Byrne Reese was previously the Product Manager of Movable Type at Six Apart, where he had also held positions as the Manager of Platform Technology and Product Manager for TypePad. Byrne is a huge supporter of the Movable Type user and developer community. He dedicates much of his time to promoting and educating people about Movable Type as well as building the tools and plugins for Movable Type that are showcased on Majordojo. He contributes regularly to open source; and he is an advocate for open protocols and standards like Atom and OpenID.
Nei on June 27, 2007, 4:28 p.m. Reply
Is there an easy way to confirm the setup is working?
Jake on June 27, 2007, 4:50 p.m. Reply
Is there any advantage at all even if this is set to 512MB or is is even worth it at that point?
Byrne Reese on June 28, 2007, 11:41 a.m. Reply
At the risk of sounding sarcastic, the best way to confirm that the setup is working is to access the application. If the setup process does not appear to finish and does not present you with a “Congrats - you have installed Movable Type” message, then something probably went awry and you should let us know.
Byrne Reese on June 28, 2007, 11:43 a.m. Reply
Any memory you can allocate to memcache is a good thing - even if it is just a couple of megabytes because memcache provides a persistence layer that works even if you don’t have FastCGI or mod_perl installed.
Mark Carey on June 29, 2007, 7:20 a.m. Reply
I will ask this question here, since I have had no luck on Pronet:
Is memcached supported “out-of-the-box” for MT’s PHP-based dynamic publishing system?
Patrick Beeson on June 29, 2007, 10:31 a.m. Reply
I just got word from the folks at Dreamhost saying they don’t allow memcached to run on shared hosting accounts.
This is a major bummer.
Byrne Reese on June 29, 2007, 12:13 p.m. Reply
@Mark - I am sorry I have not had time to reply on ProNet, but I finally have an answer for you.
Memcached does not work with PHP dynamic publishing.
That being said, I am beginning to look into a new alternative dynamic publishing model with Movable Type. This model would require the use of mod_perl of FastCGI and would also eliminate the requirement of PHP. This has the ancillary benefit of not requiring developers to implement template tags in two different languages.
Awesome. I will keep people posted. I am trying to convince engineers to revisit an old mt-view.cgi script to upgrade it to use the enhanced dynamic publishing system implemented with MT3.2.
Byrne Reese on June 29, 2007, 1:18 p.m. Reply
@Patrick - Thank you for letting us know. We will reach out to Dreamhost to see if we can’t address their concerns. I am checking with the engineers to see if we can add support for a memcache namespace parameter in the config file which will prevent users from colliding with each other. That might help.
Nei on June 29, 2007, 2:49 p.m. Reply
Pure-perl based dynamic publishing would be great.
Let’s face it, since Movable Type requires a solid Perl installation anyway, many places where it runs will easily be able to provide a persistent Perl environment as well, thus justifying any approach to have Perl handle the requests dynamically.
If I have a wish, maybe you could make the Viewer app work in such a way as a modperl handler, that no further modrewrite fumbling is required?
However, please take great care to generate appropriate ETag, Expires and Cache-control HTTP headers from the Viewer (it currently does not do this).
The only seemingly benefit PHP might serve is that it looks like a greater number of people are familiar with it; but maybe either with an EmbPerl approach or by making writing Template-handler plugins even more easy to little wizards, this could easily be fixed. (There is a certain similarity in language structure anyway.)
Additionally the extreme cost of having to maintain the code as duplicate in PHP and Perl would be alleviated as Byrne already pointed out.
I believe that doing away with the PHP/Perl hybrid is the right step for a core Perl application. Good luck!
Mark Carey on July 1, 2007, 4:48 a.m. Reply
Byrne,
Thanks for the info.
Regarding, a dynamic alternative, that is interesting too. I have often thought that it would be quite easy to take the same htaccess approach with with Perl instead of PHP, and redirect requests to mt-view.cgi instead of mtview.php. This is actually a planned extension of my Dynamic plugin, but I sometimes think I am the only person using mt-view.cgi out there. Also, I think this topic would be a good one for ProNet, when the time is right.
Jay Allen on July 1, 2007, 9:31 a.m. Reply
Please don’t do away with the PHP-based engine. Add a true (easy to use and documented) perl-based dynamic engine, but please leave the PHP engine. MT 4 is going to garner a lot of attention in corners of the web that are solidly PHP-based and having the ability to do an all PHP-based dynamic blog or site could be a real draw. I see no reason, given that MTOS is right around the corner, to throw all of that hard work away before PHP developers ever get a chance with the app.
Laura Henze on July 2, 2007, 11:41 a.m. Reply
+1
I’ve never been fond of the dynamic template engine in MT from its introduction for reasons Byrne and Mark mention and then some. That said its out there. I’ like to see dynamic page generation made pluggable via drivers and the PHP engine made into an optional driver of that.
Byrne Reese on July 2, 2007, 1:04 p.m. Reply
@Jay Allen - rest assured, PHP dynamic publishing will not go away. I want to introduce a Pure Perl DP engine in addition to what we have today.
Jay Allen on July 5, 2007, 12:08 p.m. Reply
First of all, THANK YOU. I’m just really starting to enjoy the PHP dynamic publishing engine. It was a bit of a mind bender (as Tim can attest) but once you dig into it and explore all of the smarty and ez_sql features, the mind races.
As you know, the Textura Design client extranet is built fully on a subclass of the MT dynamic core class and viewer class with added authentication mechanisms built on top. So. Much. Fun!
WRT the perl-based dynamic engine, what that poor little forgotten script has always needed most is thorough documentation. But yes, I would love to see this as an addition.
Oh and Tim, if you need someone to write PHP versions of your plugins, just let me know. I think I’ve already converted more than half of yours over time. If you’d like the code, let’s talk. You have some great ideas and plugins, but the utter lack of PHP versions really wears a fella down.
Scott Johnson on July 15, 2007, 8:48 a.m. Reply
Does anyone have benchmarks on how effective memcached improves things at different memory sizes? I’d love to see how 2mb vs 10mb vs 50mb vs 100mb caches change things for a site.
Paul Lindner on August 15, 2007, 2:55 p.m. Reply
With recent versions of memcached you should check the stats for ‘evictions’, if these are 0 you’re not running out of memory in your cache.
phil.windley.org on February 6, 2008, 7:09 p.m. Reply
On MT4.1, when I add
MemcachedServers 127.0.0.1:11211
to the mt-config.cgi file, I get this error message whenever I access MT:
Can’t locate object method “new” via package “Cache::Memcached” at lib/MT/Memcached.pm line 18.
Ideas?
Clifton on February 7, 2008, 10:02 a.m. Reply
Phil, that appears to be the error that occurs if you do not have Cache::Memcached installed. Do you have that module? MT doesn’t ship with it, as memcache is an optional feature. If you don’t have it, try installing it through CPAN or your distribution’s package manager.