Andy's Notebook

Ideas for improving my twitter traffic bot

The bot I built for keeping me up-to-date with traffic conditions on the M5 has proved handy a few times but it also suffers from a couple of drawbacks...

It only covers the M5, and the whole of the M5 at that, if someone wants updates on the M4, or  just part of the M5 there's no way for them to choose the information that's relevant to them.

TwitterFeed also introduces a time delay and even on it's most frequent setting it will only check a feed every 30 minutes - as we found out on Friday morning when we ran into a jam just north of the M4/M5 interchange only to hear our phones get the text message telling us there was a hold up!

So I've been thinking about building a new bot that would allow someone to choose the roads (and parts of)  they want to receive updates for.

The bot would allow people to send it messages about the roads they're interested in, grab the traffic feed from the Highways Agency (and perhaps other feeds) on a regular basis, and when there's something to tell people message them via twitter.

I'm thinking of something that's quite similar to mytrade's twitter bot in concept but needs to be more frequent in it's updates. The language used to communicate with the bot also needs to allow richer concepts as in stock watching terms it needs to support not just broadcasting prices, but broadcasting when a price crosses a user set threshold.

The things I'm looking at right now are:

The syntax for communicating with the bot - something similar to the syntax of mytrade's bot might be good enough but I need to have a hunt around to see what alternatives are about. (I'm not sure I like twitter's format for direct messaging and wonder whether !username might be fit better with the @username format used elsewhere on twitter, but there's not a lot I can do about that!)

Whether to use the REST API or Jabber / XMPP for communicating with twitter. The API pages suggest using Jabber / XMPP for bots that use direct messages so perhaps I'll follow their advice to begin with.

So time to brush off my coding skills and as I (hopefully) make some progress I'll add some updates here.

Some other online tools for filtering RSS

UK_M5_Traffic currently uses Yahoo Pipes to filter and manipulate the RSS feed it's based on but I came across a post on Read Write Web that lists some other online tools that might be worth exploring. ZapTXT looks to be the most interesting but won't send text messages outside the US and Canada at the moment.

In the short term I'll probably carry on using Pipes as I'm now using it's regex features to cut down the message to make it a bit more punchy and readable.

Roll your own traffic alerts using twitter and Yahoo Pipes

In the early hours of Tuesday morning there was a three lorry smash on the M5 just north of Bristol resulting in diesel and olive oil being spilt all over the road and a closed motorway. A work colleague sent Nic (my wife) a text message to let her know but unfortunately it arrived just after she had got on the motorway and so the fifteen (ish) miles to work took three hours!

During her three hours stuck on the motorway we had a few calls and of course there were questions about why wasn't there a way of letting people know about traffic problems so they could take alternate routes came up (either via signs before you get on the motorway or via text messages etc.)

A quick Google found a few sites that would send a text message when there were traffic problems but they wanted to charge a minimum of 50p a text!

So ever curious, I wondered how hard could it be to roll a free version of my own?

In the UK, the Highways Agency conveniently provides RSS feeds for traffic incidents - http://www.highways.gov.uk/traffic/11278.aspx - now all that I needed was a way to filter the feed and fire off a text message.

I've played with Yahoo Pipes in the past so it was my first port of call to filter the whole of the UK feed for just incidents affecting the M5. If you're interested the pipe's here: http://pipes.yahoo.com/andydavies/m5traffic (Tim later pointed out that I could skip this step and just use the M5 feed instead, but I might add a date and time to the message so I haven't changed from Pipes yet).

So now the feed contains just the incidents affecting the M5, what ways are there of firing off a text message? A quick email to underscore about RSS to SMS conversion and back came the suggestion of using Twitter and it's API (again, thanks to Tim).

After playing for a while with trying to drive Twitter's API from Yahoo Pipes and deciding against it, I found twitter feed. twitter feed does what it says on the tin - takes a feed and uses it to update a twitter account

So create a new twitter account - imaginatively named UK_M5_Traffic - use twitter feed to link it to the Yahoo Pipe, set my main twitter account to follow it and send me a text message when it's updated and there it is, a free automatic alert system for traffic incidents on the M5.

The first message through was a bit on the useless side as it just used the title from the item and the title doesn't contain enough information about where the problem is, but switching twitter feed to use the description fixed that. As twitter can only display 140 characters, the end of the message gets cut off but there's enough information for it to be useful.

If anyone wants to keep an eye on the M5, the final result is here: http://twitter.com/UK_M5_Traffic