Action Streams Recipe Developer Guide

The Action Streams plugin collects authors’ actions performed in remote services. Plugin developers can add support to Action Streams for new web services, often with no Perl code required at all.

Making a plugin with the registry

In Movable Type, plugins are defined using a file called config.yaml instead of Perl code. When Movable Type loads the plugin, the settings the plugin defines in config.yaml are integrated into what’s called the registry. The registry is a tree of settings that informs Movable Type what settings, pages, and objects are available.

To define your plugin, make a new directory in the plugins directory. Inside it, create a file called config.yaml, and enter all the basic plugin settings. For example:

id:   ExampleService
key:  ExampleService
name: Example Service
description: An example Action Streams service.
version: 1.0
author_name: Mark Paschal
author_link: http://www.example.com/mark/
plugin_link: http://www.example.com/plugins/example-service/

These settings define a plugin called “Example Service” with all the specified settings. Consult the Movable Type developer documentation for further help on creating plugins.

Defining a profile service

Every action and stream belongs to a service, so unless you’re adding a stream for a service that already exists in Action Streams, you need to define one of those first. Usually a service is a web site at which an author has an account. Authors can use the Action Streams plugin to make a sidebar list of their accounts on all the services for which they enter their accounts.

Profile services are defined in the profile_services section of the registry. Because Movable Type merges your config.yaml into the registry, you specify additional services by adding your own profile_services section to your plugin’s config.yaml. For example:

profile_services:
    example:
        name: Simple Example Service
        url: http://www.example.com/people/{{ident}}
    complex:
        name: Complex Example Service
        url: http://big.example.com/{{ident}}/profile
        ident_label: User ID
        ident_example: 12345
        ident_suffix: /profile
        service_type: example
        icon: example.png

This defines two services. Each service has its own key by which it’s known in the registry. These services’ keys are example and complex. The data you can set for a service are:

`name` (required)

The human-readable name of the service. This name is used for display and for ordering the list of services.

`url` (required)

The URL to a user’s profile on the service. This is the URL used in the author’s list of services.

The string {{ident}} will be replaced by the author’s identifier for this service. Only one identifier per service is currently supported.

`ident_label`

A label to describe what the author’s identifier on the service is. If not given, the label Username is used. For example, if the user identifier on the service is a number, you might specify User ID here to indicate the author should enter a number. For the AIM service, the label is Screen name.

`ident_example`

A label to show what the identifiers for the service generally look like. For example, if the user identifier is a number, you might specify 12345. For the Flickr service, the example identifier is 36381329@N00.

`ident_suffix`

A label to show after the field in which the identifier is entered. Typically this is used to further suggest that the one’s subdomain on a service is one’s identifier. For example, the Vox service’s suffix is .vox.com.

`service_type`

The kind of web service this profile is about. Authors can list their accounts for a certain type of services using the type attribute of the OtherProfiles tag. Any value is allowed here, but the established service types are:

contact (AIM, Yahoo! Messenger, etc)
blog (Vox, Tumblr, the Website service, etc)
photos (Flickr, Smugmug, etc)
video (Vimeo, YouTube, etc)
links (Delicious, Digg, etc)
status (Pownce, Twitter, etc)
network (Facebook, MySpace, etc)

`icon`

The location of the service’s icon. This can be a complete URL to the image. If not a URL, icon should be a path to the image relative to your plugin’s mt-static directory. For example, if you install your example.png at mt-static/plugins/ExampleService/example.png, your icon setting should be example.png.

Defining stream recipes for a service

Once a service is defined in profile_services, you can make recipes for collecting its action streams. These recipes go in the action_streams section of the registry. For example:

action_streams:
    example:
        posts:
            name: Posts
            description: Posts you posted on the example service
            html_form: '[_1] posted <a href="[_2]">[_3]</a>'
            html_params:
                - url
                - title
            url: http://www.example.com/people/{{ident}}/posts
            atom:
                thumbnail: media:thumbnail/child::text()

Each service has its own section, labelled with the same key as in the profile_services section. Inside that, you can define one or more recipes for how to collect actions. As you can see, recipes have three parts:

URLs that indicate where to find action content.
Collectors that specify how to turn the resource at the stream’s URL into action data. Action data is primarily a title, a link, and a unique identifier for each action, but you can specify additional data fields in your recipe.
Forms that describe how to turn the action data back into HTML. Authors can display however they like with template code, but in the application and by default, actions in this stream will display as you specify in the recipe.

Together, these parts define the recipe.

Picking a stream ID

The word you use as the registry key when defining a stream is its ID. While you can use any term here, if the stream you’re defining is analogous to a stream available from another service, use the same ID as the other service’s stream does.

Template authors can list similar actions across services by using the stream attribute on the ActionStreams tag in absence of the service attribute. The value template authors will use with stream will be this ID, so in order for your (say) photos stream to be included when a user lists all posted photos across all services, use photos for your stream ID.

Some of the standard stream IDs used in the standard provided streams are:

links, web pages or resources that have been bookmarked (as in Delicious, Digg, Magnolia)
photos, photos posted by the author (as in Flickr, Picasa on the Web, Smugmug)
favorites, assets and objects of others’ that the author saved as a favorite (as in Vox, Twitter, YouTube)
achievements, awards won by the author (as in Steam or Kongregate)
videos, videos posted by the author (as in Viddler, Vimeo, YouTube)

Picking the stream resource

Typically, modern web services will provide a web feed or XML resource for their members that rolls up their recent activity on the site. Sometimes a service only provides an HTML page, but Action Streams can work with them too. This is the resource you will specify as your stream’s url.

The URL for the resource needs to include the user’s identifier in the URL. In some extreme cases, you may even have to change what identifier your profile service uses in order to get the stream resource. Normally the same user name or ID is in the URL, though.

Three schools of collection

Most services don’t yet publish their members’ actions as such, so even if you can collect from a nice standard web feed, the feed content will need some massaging. There are three main kinds of collectors at your disposal: XML feeds, HTML scrapers, and custom collectors.

XML feeds

It’s easy to collect data from XML feeds. Action Streams accepts recipes using XPath syntax. For example, you might collect data from Delicious’ RSS feeds with an xpath recipe:

delicious:
    links:
        name: Links
        description: Your public links
        # ...
        url: 'http://delicious.com/rss/{{ident}}'
        identifier: url
        xpath:
            foreach: //item
            get:
                created_on: dc:date
                title:      title
                url:        link

This specifies that each item in the feed is an XPath //item, meaning an item tag anywhere in the document. Within each item, you can find values for the created_on, title, and url fields using the given XPath selectors. The identifier: url statement indicates to use the url value as the action’s unique identifier.

As many services provide content as standard web feeds, it’s actually even easier to collect from those. Instead of xpath, specify your recipe as rss or atom, with any additional or special fields as additional XPath selectors:

delicious:
    links:
        name: Links
        description: Your public links
        # ...
        url: 'http://delicious.com/rss/{{ident}}'
        identifier: url
        rss:
            created_on: dc:date

If there are no special fields to collect, you can also specify the recipe as simply 1, as in this recipe for Jaiku:

jaiku:
    jaikus:
        name: Jaikus
        description: Jaikus you posted
        # ...
        url: 'http://{{ident}}.jaiku.com/feed/atom'
        atom: 1

The values of each entry’s ID field (guid in RSS and id in Atom) are then used for actions’ identifiers.

HTML pages

When feeds aren’t available, HTML will do. Action Streams reads HTML pages using Web::Scraper, an HTML scraping library inspired by the Ruby scrapi library. For example, the Iwtst plugin reads your iwanttoseethat.com profile page with this definition:

iwtst:
    want:
        name: Movies
        description: The movies you want to see
        # ...
        url: 'http://iwanttoseethat.com/people/{{ident}}'
        identifier: url
        scraper:
            foreach: 'div.seethat div.seethat_yes'
            get:
                url:
                  - a
                  - @href
                title:
                  - a
                  - TEXT

Instead of XPath, here we can use CSS selector syntax. (For more complex constructions, you can use XPath selectors as well.) This recipe says each action is a div with class seethat_yes inside a seethat div. The action’s url field is then the value of the href attribute of the a tag inside, while the title field is the text linked inside that a tag.

Custom collectors

If you aren’t pulling web data, or you can’t quite articulate the set of selectors to pull out the data you want, you can implement your own action collector as a Perl class. Specify what class implements your stream as the class option:

complex:
    posts:
        name: Posts
        description: Posts you posted on the complex example service
        # ...
        class: Example::Event::ComplexPosts

Your class should subclass the plugin’s ActionStreams::Event class, and implement your particular content-finding behavior. As the XML and HTML recipes are really the default behavior of an ActionStreams::Event object, the above techniques are easily accessible from your class too. See some of the ActionStreams::Event subclasses included with the Action Streams plugin for examples.

Formatting actions for display

While bloggers can use the provided template tags to customize how actions are published to the blog, recipes define how actions are displayed by default: when using the mt:StreamAction tag and when displayed in the application. This display is governed by the html_form and html_params options, as in the Delicious recipe:

delicious:
    links:
        name: Links
        description: Your public links
        html_form: '[_1] saved the link <a href="[_2]">[_3]</a>'
        html_params:
            - url
            - title
        # ...

The html_form setting is a pattern in maketext format, making it easy to . Each replacement token is numbered. The first replacement token, [_1], is always the name of the MT user whose action is being displayed. The next tokens represent the fields you specify in the html_params list (in this case, url and title).

Note that the result of formatting an action is, in fact, HTML. This markup will be returned verbatim when used with the mt:StreamAction tag. For display in the Movable Type application, the HTML is removed; Delicious links are the text “Author name saved the link title” without the hyperlink, for example.

Cleaning up after your stream

The provided collectors described above may not leave the action data in the correct displayable format. For example, the regular Twitter tweet feeds include the authors’ usernames with the tweets. In order to use the feeds, the Twitter stream needs to remove the names, so the tweet text is only the tweet text. How can it remove the usernames from the collected actions?

Movable Type provides a system for defining callbacks, or code hooks that are run at certain points of a process. The Action Streams plugin provides several callbacks at various stages of stream event collection. You will have to use Perl code to write these callbacks, but as you’re doing very small things, the code is usually simple.

For example, the Twitter stream uses the pre_build_action_streams_event callback to modify the actions after they’re collected by the normal feed collector, but before they are saved. The callback is defined in the registry thus:

callbacks:
    pre_build_action_streams_event.twitter_tweets: \
        $ActionStreams::ActionStreams::Plugin::fix_twitter_tweet_name

The stream callback code is then defined in ActionStreams::Plugin as:

sub fix_twitter_tweet_name {
    my ($cb, $app, $item, $event, $author, $profile) = @_;
    # Remove the Twitter username from the front of the tweet.
    my $ident = $profile->{ident};
    $item->{tweet} =~ s{
        \A          # start of the string
        \s*         # any whitespace
        \Q$ident\E  # the Twitter username
        : \s*       # followed by a colon and whitespace
    }{}xmsi;
}

The Action Streams callbacks available to you are documented later. See the Movable Type developer documentation for more about defining callbacks.

Stream recipe option reference

Recipes describe how to collect and display users’ actions on profile services. They are found in the action_streams section of the registry, in sections named the same as the related service in the profile_services section.

The options you can set for action streams are:

`name` (required)

The name of the action stream. This is displayed to authors when they enter their profile service identifier, where they are given the option of collecting the stream.

`description` (required)

The description of the action stream. This is displayed to authors with the stream’s name when they enter profile identifiers.

`fields`

The list of additional fields supported by actions of this stream.

Every stream already has a set of standard fields. You need only specify additional fields beyond these. The standard fields are:

`title`

The name of the page/asset/resource the action is about.

`url`

The web address of the page/asset/resource the action is about.

`thumbnail`

The web address (URL) of a thumbnail image of the page/asset/resource the action is about.

`identifier`

A string that uniquely identifies this action in its particular stream.

`created_on`

When the action was taken. This is not necessarily when the page/asset/resource the action is about was created; when a link is saved or a video is favorited, for example.

`modified_on`

When the action was last altered. Many actions occur only once and will never change, but some may (such as if the author changes the rating or tags for a saved link). Action Streams will update this timestamp automatically when the action object is changed.

`html_form`

The formatting string for formatting an action for display.

Numbered replacement tokens, beginning with [_ and ending with ], are replaced with the author’s name and the values of the fields named in html_params. For example, [_2] in the html_form will be replaced with the value of the second field specified in html_params.

`html_params`

The list of additional fields to replace into html_form when displaying an action. The values of the event fields named here are replaced into the numbered replacement tokens of html_form to produce the HTML version of an action.

If html_form has more tokens than there are fields specified in html_params, the nickname of the author whose action is being rendered is provided for your convenience. That is, if you omit the first field, [_1] in your html_form will automatically be replaced with the action’s author’s name.

If your event class is implemented as a Perl class, you can also use the names of any additional methods that class implements in your html_params.

`url`

The URL from which to collect action data. The token {{ident}} will be replaced with the author’s identifier on that service.

`identifier`

The name of the field to use as the unique identifier. Multiple fields can be specified by separating their names with commas (for example: title,created_on).

If not given, the collection recipe should collect an identifier field from the action source. If an identifier field is not collected, each action will be considered unique, regardless of the content of the other fields.

`xpath`

The XPath recipe with which to collect action data. The recipe options for an XPath recipe are:

`foreach` (required)

The XPath selector that selects the nodes of the resource document that represent actions. All the get selectors are then applied to each node to find the action data.

`get` (required)

A set of XPath selectors, keyed on the names of the action fields they describe. To find the value of each field, the indicated selector is applied to each node as determined by the foreach selector.

`rss`

The RSS recipe with which to collect action data. RSS recipes collect data from the standard RSS entry content. If child options are specified, each is used as an XPath selector (as in an xpath recipe’s get option) to collect a field from the RSS item nodes.

`atom`

The Atom recipe with which to collect action data. Atom recipes collect data from the standard Atom feed content. If child options are specified, each is used as an XPath selector (as in an xpath recipe’s get option) to collect a field from the Atom entry nodes.

`scraper`

The Web::Scraper recipe with which to collect action data. The options for a Web::Scraper recipe are:

`foreach` (required)

The CSS or XPath selector that selects the nodes of the resource document that represent the actions. All the get selectors are then applied to each node to find the action data.

`get` (required)

A set of Web::Scraper selectors, keyed on the names of the action fields they describe. Each selector is applied to each action node, as determined by the foreach selector, to find the action data.

Each Web::Scraper selector is a two-element list:

The CSS or XPath selector to use to find a child node.
@attribute or TEXT to select an attribute value or the text content of the node.

`class`

The name of the Perl package to use to represent actions in this stream. The Perl package should be a subclass of ActionStreams::Event. See the ActionStreams::Event perldoc for information on defining an action stream in Perl code.

Stream recipe callback reference

Action Streams specifies several callbacks for you to customize the creation of your action streams and the collection of their events.

Note that Movable Type’s standard object callbacks are also available for the ActionStreams::Event objects representing actions.

`pre_add_profile.``type`

Fires before an author’s new profile is added. type is the ID (registry key) of the service being added. The parameters passed to your callback routine are:

$app, the current MT::App through which the profile is being added
$user, the MT::Author whose profile is being added
$profile, a hashref containing the information about the profile being added:
- type, the ID of the profile service
- ident, the user ID on the remote service
- label, a user-facing description of the profile
- uri, the URL of the profile page
- streams, a hashref with keys that are the stream IDs of the streams the author has selected to collect; all values are set to 1

`post_add_profile.``type`

Fires after an author’s new profile is added. The same parameters are passed to your callback routine as to a pre_add_profile routine.

`pre_remove_profile.``type`

Fires before a profile is removed. The parameters passed to your callback routine are:

$app, the current MT::App through which the profile is being removed
$user, the MT::Author whose profile is being removed
$type, the ID of service corresponding to the profile being removed
$ident, the user ID on the remote service for the profile being removed

`post_remove_profile.``type`

Fires after a profile is removed. The parameters passed to your callback are the same as to a pre_remove_profile callback.

`pre_action_streams_task`

Fires before the scheduled task to collect stream events is performed. The parameters to your callback routine are:

$mt, the current MT instance through which the stream events will be collected

`post_action_streams_task`

Fires after the scheduled task to collect stream events is completed. The parameters to your callback are the same as for pre_action_streams_task.

`pre_update_action_streams_profile.``type`

Fires before the streams for a particular author’s profile are processed. For example, this callback fires once before an author’s Twitter streams (both the tweets and favorites streams) are collected. The parameters passed to your callback routine are:

$mt, the current MT instance through which the stream events will be collected
$author, the MT::Author whose streams are being collected
$profile, a hashref containing all the information about the profile; this is the same as the $profile argument to the pre_add_profile callback

`post_update_action_streams_profile.``type`

Fires after the streams for a particular author’s profile are processed. The parameters passed to your callback routine are the same as to pre_update_action_streams_profile.

Note that in the current implementation, action streams are collected asynchronously, whereas this callback is fired synchronously after the collection jobs are queued. The new actions from the author’s profile will not have been collected when this callback occurs.

`pre_build_action_streams_event.``type`

Fires after a stream event is collected, but before it’s put into the ActionStreams::Event object. The parameters given to your callback routine are:

$mt, the current MT instance through which the stream event was collected
$item, a hashref containing the data collected for this event
$event, the ActionStreams::Event into which the event data will put; this is either a new empty ActionStreams::Event, or the previous version of this event as determined by the stream recipe’s identifier
$author, the MT::Author whose stream is being collected
$profile, a hashref containing all the information about the profile; this is the same as the $profile argument to the pre_add_profile callback

`post_build_action_streams_event.``type`

Fires after a stream event is composed, but before it is saved. The parameters passed to your callback routine are the same as to pre_build_action_streams_event.

Action Streams Recipe Developer Guide

Making a plugin with the registry

Defining a profile service

name (required)

url (required)

ident_label

ident_example

ident_suffix

service_type

icon

Defining stream recipes for a service

Picking a stream ID

Picking the stream resource

Three schools of collection

XML feeds

HTML pages

Custom collectors

Formatting actions for display

Cleaning up after your stream

Stream recipe option reference

name (required)

description (required)

fields

title

url

thumbnail

identifier

created_on

modified_on

html_form

html_params

url

identifier

xpath

foreach (required)

get (required)

rss

atom

scraper

foreach (required)

get (required)

class

Stream recipe callback reference

pre_add_profile.type

post_add_profile.type

pre_remove_profile.type

post_remove_profile.type

pre_action_streams_task

post_action_streams_task

pre_update_action_streams_profile.type

post_update_action_streams_profile.type

pre_build_action_streams_event.type

post_build_action_streams_event.type

Getting Started

Guides

Reference

Solutions

`name` (required)

`url` (required)

`ident_label`

`ident_example`

`ident_suffix`

`service_type`

`icon`

`name` (required)

`description` (required)

`fields`

`title`

`url`

`thumbnail`

`identifier`

`created_on`

`modified_on`

`html_form`

`html_params`

`url`

`identifier`

`xpath`

`foreach` (required)

`get` (required)

`rss`

`atom`

`scraper`

`foreach` (required)

`get` (required)

`class`

`pre_add_profile.``type`

`post_add_profile.``type`

`pre_remove_profile.``type`

`post_remove_profile.``type`

`pre_action_streams_task`

`post_action_streams_task`

`pre_update_action_streams_profile.``type`

`post_update_action_streams_profile.``type`

`pre_build_action_streams_event.``type`

`post_build_action_streams_event.``type`