TweetStream: Ruby Access to the Twitter Streaming API

by Michael Bleigh on September 22, 2009 • filed under open-source twitter streaming api release announcementcomment

Twitter’s Streaming API is one of the most exciting developments in the Twitter API in some time. It gives you the ability to create a long-standing connection to Twitter that receives “push” updates when new tweets matching certain criteria arrive, obviating the need to constantly poll for updates. TweetStream is a Ruby library to access the new API.

Installation

Installation of TweetStream is simple, it’s available as a gem on GitHub and Gemcutter.org. To install it from GitHub:

gem sources -a http://gems.github.com
gem install intridea-tweetstream

To install it from Gemcutter:

gem sources -a http://gemcutter.org
gem install tweetstream

Usage

TweetStream creates a long-standing HTTP connection to Twitter, so unlike other Twitter libraries you don’t simply run it once and deal with the results. Instead, you provide a block that will be yielded to with each new status that arrives. The most basic example is:

require 'rubygems'
require 'tweetstream'

TweetStream::Client.new('user','pass').sample do |status|
  puts "[#{status.user.screen_name}] #{status.text}"
end

This will provide you with a small sample snapshot of all of the updates being posted to Twitter at this moment and print them to the screen. There are also methods available to track single-word keywords as well as the updates of a specified list of user ids (integers, not screen names). You can do that like so:

# Track the terms 'keyword1' and 'keyword2'
TweetStream::Client.new('user','pass').track('keyword1', 'keyword2') do |status|
  puts "[#{status.user.screen_name}] #{status.text}"
end

# Track users with IDs 123 and 456
TweetStream::Client.new('user','pass').follow(123, 456) do |status|
  puts "[#{status.user.screen_name}] #{status.text}"
end

Handling Deletion/Limit Notices (Updated in 0.1.4)

Sometimes the Streaming API will send messages other than statuses.
Specifically, it does so when a status is deleted or rate limitations
have caused some tweets not to appear in the stream. To handle these, you can use the on_delete and on_limit methods. Example:

TweetStream::Client.new('user','pass').on_delete{ |status_id, user_id|
  Tweet.delete(status_id)
}.on_limit { |skip_count|
  # do something
}.track('intridea') do |status|
  # do something with the status like normal
end

Daemonization

One of the most useful features of TweetStream is its built-in daemonization functionality. This allows you to create scripts that run in the background of your machine rather than taking up an active process. To create a daemon script, you simply use TweetStream::Daemon instead of TweetStream::Client. Here’s an example:

require 'rubygems'
require 'tweetstream'

# The third argument is an optional process name.
TweetStream::Daemon.new('user','pass','tracker').track('keyword1','keyword2') do |status|
  # Do something like dump the status to ActiveRecord
  # or anything else you want.
end

If you were to place the above code in a file called tracker.rb you could then run ruby tracker.rb to see a list of daemonization commands such as start, stop, or run.

TweetStream is a simple wrapper on the Streaming API, but with built-in daemonization provides powerfully flexible means of accessing the Twitter Streaming API using familiar Ruby tools. More complete code documentation is available at rdoc.info.

Update: I overlooked the deletion and rate limit notices when I wrote the initial version of the gem. As of version 0.1.4 these are handled properly.

Comment | 
blog comments powered by Disqus

Words we've written view all blog posts »

Featured Article

Intridea Insider: Ping Yu

by Renae Bair on July 30, 2010

Ping came on board with Intridea during our infancy. He was employee #4, and has been part of the vision and culture of our company from the beginning. He has helped to grow Intridea from a group of four guys working at their dining room tables, to a company of 40+ Ruby and Rails developers and designers that span continents and multiple time zones. His work as Director of Asian Operations aims to bridge our Asian team of Rubyists with our centralized team in the states. "I believe in helping to extend Intridea into China. There is so much talent in China, it feels like a natural thing to do. Intridea has a culture of quality, and I want to bring that to these developers. They love Ruby, and they have so much energy."

Continue reading »

Recent Blog Posts

Intridea Insider: Ping Yu

by Renae Bair on July 30, 2010

Talkin' About Upgrading to Rails 3

by Michael Bleigh on July 26, 2010

A To-Go Plate of Ruby Midwest

by Michael Bleigh on July 19, 2010