Toluu

Twitter Technology Blog

We Got DataOctober 24

There are lots of ways to get data into and out of Twitter. So many, in fact, that it's become a bit confusing as to what a developer's options are. We'd like to clear that up.

REST API

If you want to interact with Twitter on behalf of an individual user or a small group of users, your best bet is our REST API. It's perfect for making updates, retrieving timelines of tweets, marking tweets as favorites, and so forth — most any feature you'll find on twitter.com has a corresponding method in the REST API. Desktop applications like Twitterrific and Twhirl use our REST API, as do plenty of web-based and mobile device applications, widgets, and scripts.

Search API

If you want to programatically retrieve tweets about a particular topic, check out our Search API. We provide a lot of flexibility to support a variety of queries. You can filter by criteria like location, hashtag, dates, language, and more. We're seeing more great applications powered by Twitter Search every day.

Data Mining Feed

If you're interested in doing research on the Twitter community, we provide a data mining feed to meet that need. The feed

Summize and The Future of the Twitter APIJuly 16

Earlier today we announced that Twitter has acquired Summize, the leading Twitter search service. I'd like to explain how this change will impact the Twitter developer community.

Summize — now search.twitter.com — has their own API that a number of applications are already using. Over at the Twitter Development Talk group I've been encouraging developers with search and data-mining needs to investigate the Summize API. Developers can now trust that Summize has the entire corpus of public Twitter updates at their disposal for search going forward.

For the time being, the Summize API will live at search.twitter.com as a separate entity from the main Twitter API. We'll try to get the two APIs as coordinated as possible in the short term, but merging Summize's architecture with ours will be a gradual process. That said, we have a clear opportunity for cleanly merging the two APIs.

The Twitter API has much room for improvement. As it's grown over the last year and a half, our API has become increasingly less RESTful. Resource names need to

You've Got Q's, We've Got A'sMay 30

We had a lot of feedback on our architecture update. In the spirit of continued openness and transparency, I'd like to address a number of the questions that came up in the comments on that post.

Donnie asks if we're making a slow exodus from Ruby, and Scabr asks if our key problems were Ruby problems.  We've got a ton of code in Ruby, and we'll continue to develop in Ruby with Rails for our front-end work for some time.  There's plenty to do in our system that Ruby is a great fit for, and other places where different languages and technologies are a better fit.  Our key problems have been primarily architectural and growing our infrastructure to keep up with our growth.  Working in Ruby has been, in our experience, a trade-off between developer speed/productivity and VM speed/instrumentation/visibility.

RBL asks how we're instrumenting our application. We've used DTrace on a couple of occasions, but for the most part we do a lot of print-style logging to a combined syslog. We also recently ad

Twittering About ArchitectureMay 22

Here at Twitter HQ, we're not blind to the flurry of discussion over the past weeks about our architecture. For many of our technically-minded users, Twitter downtime is an opportunity to muse about what the source of our problems might be, and to propose creative solutions. I sympathize, as I clearly find our problems interesting enough to work on them every day.

Part of the impetus for this public discussion extends from the sense that Twitter isn't addressing our architectural flaws. When users see downtime, slowness, and instability of the sort that we've exhibited this week, they assume that our engineering progress must be stagnant. With the Twitter team working on these issues on and off for over a year, surely downtime should be a thing of the past by now, right? Shouldn't we be able to just "throw more machines at it"?

To both rhetorical questions, the answer is "not quite yet". We've made progress, and we're more scalable than we were a year ago, but we're not yet reliably horizontally scalable. Why? Because there are significant portions of our system that need to be rewritten to meet that goal.

Twitter is, fundamentally, a messaging system. Twitter was not architected as a messaging system, however. For expediency's sake, Twitter was built with technologies and practices that are more appropriate to a content management system. Over the last year and a half we've tried to make our system behave like a messaging system as

How to Build a Twitter AgentFebruary 15

Dominiek ter Heide had a dream. A dream of a "Twitter Service that will allow us to track the Blogosphere." So he built it, and blogged about the process! Dive in to his post to learn about building Twitter bots that talk our service via Jabber/XMPP.

If Ruby's your thing, you'll also learn about some great tools for XMPP, managing background tasks, and more in Dominiek's post. If it's not, you can take much of Dominiek's approach and apply it in your favorite language with XMMP libraries like python-xmpp for Python, XMPP.NET for C#, Gloox for C++, Smack for Java, xmpp4moz on the Mozilla platform, or one of many other libraries out there.

Jabber is great for way more than just chatting with your friends, and until now it's been up to talented developers like Dominiek to muddle their way through building Twitter bots and applications with the protocol. I'd like to put together an "official" guide to developing Jabber/XMPP applications for Twitter in the near future. Until then, soak up the knowledge that our great developer community