Building Action Streams

As my LiveJournal friends know, I’m reading Bowling Alone, Robert Putnam’s 2001 book about the decline in civic engagement in the US since the generation that fought World War II. Without going into his arguments, he shows that the behavior we expect from a civil society has been evaporating since roughly the dawn of television, leaving us with brittle institutions and less resilient lives.

There are both a mighty need and a grand opportunity for us to knit our society back together, and I expect us to use the internet to do that. Putnam mentions the idea that the internet lets us connect in ways we haven’t before, but rightly views the utterly unproven possibility with skepticism. I’ve yet to adequately articulate myself on the topic, and I’m still not able to do so here, so instead let’s discuss a meager tilt I took at that windmill: the new Action Streams plugin for Movable Type 4.1.

What?

Today we’re releasing (as we discuss on Six Apart News) a new plugin for Movable Type 4.1 called Action Streams. It provides a framework for collecting your actions from services around the web into one place for you to share back out as you see fit. The most obvious example might (might) be my blog, where I’ve been using Action Streams since I started building it, but you can also see David’s, Byrne’s, Michael’s, or the team’s group stream on movabletype.org.

The plugin lets you enter your profiles from other sites, publishing the list as a sidebar widget. (This function is available on Vox and TypePad as well, and this MT implementation was originally a plugin Ben wrote.) I’ve added a new step where, given instructions built into the plugin, MT will collect your activity into a stream you can share on your own site.

Compliance

All of us are in an interesting position: we have all the technologies and the vocabulary to make open social networking tools, we just have to do it. I have great respect for the goals of DiSo: when we already have great wheels, it’s frustrating to have to reinvent them. In a similar vein, we built Action Streams from the start on existing technologies.

For Action Streams to collect your activity, it has to know where your other profiles are; once you enter them, the plugin helps you share those, too. Using the example template code or the widget in the provided template set, the list is automatically marked up for discoverability with rel="me" via the Microformat XFN as we’ve already done in TypePad and Vox.

Action Streams speaks Atom natively: services that provide your actions as an Atom feed are the easiest ones to add. We also made it easy to republish your actions, with both an example Atom feed template and an Atom feed in the template set. We also added hAtom markup to the HTML versions of streams.

Plugging in new services

Action Streams lets you identify all your profiles on a long list of other services, but no one of us has the time to catalog every last service you want to use. (Ideally, as action streams become more common, more services will directly provide your actions as real, reusable Atom content.) So I designed Action Streams to make it simple to drop in new services and stream definitions.

With Movable Type 4’s registry, other plugins can add new services and streams to Action Streams’ framework. Included in the Action Streams distribution is an example “Iwtst” plugin that shows how to add support for a site, in this case for I Want To See That!, by writing only a YAML file. (Plugin authors who need more flexibility can still write Perl code.)

Here’s how the Iwtst plugin defines the profile service:

profile_services:
    iwtst:
        name: I Want To See That!
        url: http://iwanttoseethat.com/people/%s
        icon: images/iwtst.png

(The icon is one I made, and included in mt-static/plugins/Iwtst.) And here’s the result:

Defining streams is a little harder. First you have to understand the parts of an action:

• Actions are in streams, which are defined for services. A service like Vox can have multiple streams (posts and favorites).
• Actions have fields and patterns. Almost all actions on the web will have a title and a URL, so those fields are already defined for you, but you can define more. Content patterns form the values of the fields into the text of the action (“Mark saved the link blah”).
• Actions also have URLs; Action Streams will fetch the resource at the stream’s URL to find the content.
• Identifiers are the unique data that describe actions, to prevent duplication and allow updating changed actions.

Only when we have all these parts can we define the recipe: how to turn the web content at the stream’s URL into the action content.

Three schools

As most services don’t yet publish your actions as such, Action Streams provides tools to collect your actions from XML and HTML documents. The recommended method is to pull data from XML feeds, mapping nodes in the document to action fields using XPath. Here’s how your public del.icio.us links are pulled from your feed (in this case, an RSS feed):

delicious:
    links:
        name: Links
        description: Your public links
        html_form: '[_1] saved the link <a href="[_2]">[_3]</a>'
        html_params:
            - url
            - title
        url: 'http://del.icio.us/rss/{{ident}}'
        identifier: url
        xpath:
            foreach: //item
            get:
                created_on: dc:date/child::text()
                title: title/child::text()
                url: link/child::text()

As more services publish action-oriented feeds, more stream definitions would look more like the definition for Ma.gnolia links: the atom says “this is an Atom feed,” but we supply an XPath hint for where to find the tags.

magnolia:
    links:
        name: Links
        description: Your public links
        html_form: '[_1] saved the link <a href="[_2]">[_3]</a>'
        html_params:
            - url
            - title
        url: 'http://ma.gnolia.com/atom/full/people/{{ident}}'
        identifier: url
        atom:
            tags: "category[@scheme='http://ma.gnolia.com/tags']/@term"

When feeds aren’t available, HTML will do. The Iwtst plugin reads your profile page’s HTML using Tatsuhiko Miyagawa’s scrapi-inspired Web::Scraper, with this definition:

iwtst:
    want:
        name: Movies
        description: The movies you want to see
        html_form: '[_1] wants to see <a href="[_2]">[_3]</a>'
        html_params:
            - url
            - title
        url: 'http://iwanttoseethat.com/people/{{ident}}'
        identifier: url
        scraper:
            foreach: 'div.seethat div.seethat_yes'
            get:
                url:
                  - a
                  - @href
                title:
                  - a
                  - TEXT

Web::Scraper allows you to specify resources in either XPath or, as here, CSS syntax. So the scraper section says, “Look for all the divs of class seethat, then all the divs of class seethat_yes. Find an a inside, and collect the href attribute as the url and the text content of the a as the title.”

Even as easy as adding 90% of services is, sometimes you just have to write code. (So really there’s a fourth school. Sorry.) The Flickr favorites stream is an example. As there’s no easy feed for those actions, we instead use Flickr’s API to find your public favorites. (You can look at the ActivityStreams::Event::Flickr module to see that code.)

A first step

This is the first component out of a long series of new networking technologies that we’re seeing come online. It’s disappointing that third parties have had to pick up the slack, when companies like us can use our resources to do this productization work. As the pieces click into place, they will let us connect with each other not only in ways we haven’t yet seen, but in ways that are obvious, but aggravating not to already have.

It looks to be a fun year!