Behind the scenes at #cpd23: automating (some of) the registration process

A couple of weeks ago, I asked for thoughts about ways of automating the process of adding new subscriptions to a single RSS feed drawn from multiple sources.  This was to try and streamline the process of managing registrations for 23 Things for Professional Development. I seem to have come up with something that works, albeit in a somewhat ungainly fashion, so I thought I'd write it up.
Here's a diagram of what's what.  Red lines and text are manual steps done by the cpd23 team, and the blue lines and text are the automated bits.  The yellow stars show the outputs that are intended for general use.  If you can't see the image for whatever reason don't worry - I explain all the steps below.

Icons by Mysitemyway Design Team via iconfinder.com

The aims of the system are

1. to generate a tagged list of participants, so that participants and others can find blogs that interest them, based on geographical region and professional sector
2. to create a feed of recent posts by participants to show on the cpd23 homepage
3. to create a combined feed of all cpd23 participants' posts, to which anyone who is interested can subscribe

This is achieved in the following way:

1. Participants register their blogs using a Google Docs form (embedded into this blog post).The data submitted into a Google Docs form is stored in a Google Docs spreadsheet, which can be accessed and edited.  
2. Cpd23 team members manually visit each blog that is registered, and (if the link submitted is valid and the blog is a personal cpd blog) they add it to Delicious, including tags for place, sector, year and an alphabetical tag for the first letter of the blog title.  (I use the handy Delicious bookmarklet for this.)
• Delicious provides scripts that power the 'recently signed-up' widget on the cpd23 blog, and the participants list on the participants page.  There is a limit to the number of links that Delicious will provide (100 at a time), which is why the alphabetic tags are added to the bookmarks: the participants list is powered by a script for each letter of the alphabet.
3. An ifttt task takes all new bookmarks added to Delicious with the tag '2012' and submits them to the Google Reader folder 2012. Where possible, Google Reader can automatically extract an RSS feed from the link given.
• Cpd23 team members periodically update the 'Participants posts' widget on the cpd23 blog, which runs using the Blogger blogroll widget.  It's easy to add new subscriptions in the 2012 folder from Google Reader.
4. Another ifttt task takes all new posts in the Google Reader 2012 folder, and bookmarks them in Diigo, adding the tag 'cpd23feed'. (It could bookmark them in Delicious instead, but I wanted to keep the Delicious account just for links to blogs, and not confuse matters by adding lots and lost of links to posts there).
• Diigo provides RSS feeds of users' tags, so an RSS feed of all new posts from cpd23 participants can be made available. The format of Diigo RSS feeds is: http://www.diigo.com/rss/user/[username]/[tag]
5. And that's it.

  by  roberthunt1987 

Benefits of this system:

• It minimizes the number of manual steps - we only need to get involved at two points in the process
• It enables us to offer the 'added value' service of a combined RSS feed, on top of the Blogger blogroll and the tagged Delicious participants list.

Disadvantages of this system:

• It relies on a lot of services running correctly
• Time lag: the ifttt tasks, and the feeds from Delicious and Diigo update periodically, so new blogs and posts aren't available everywhere immediately.  The lag is usually up to an hour.
• Ifttt only recognises new tags, so if Delicious recognises a 2011 participant blog such that we're just adding a new 2012 tag to an existing bookmark that blog has to be added to Reader by hand.
• Google Reader doesn't always correctly find the feed of a link submitted from Delicious.
• The resulting RSS feed, because it's a feed of bookmarks not posts, only shows links and not post text.  This really isn't ideal, but it's the best I could come up with.